Displaying HDMI content in a tiled window

ABSTRACT

In response to receiving user-interface activity information from a portable electronic device that specifies user selection of a content source, an audio/video (A/V) hub provides a request for high-definition multimedia-interface (HDMI) content to the content source based on the user selection. When the A/V hub receives the HDMI content from the content source, the A/V hub provide the HDMI content and display instructions to an A/V display device as frames with the HDMI content are received from the content source, so that the HDMI content is displayed in a tiled window on the display in the A/V display device while other HDMI content from another content source is displayed on the display.

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims priority under 35 U.S.C. 120 as aContinuation-in-Part of U.S. patent application Ser. No. 15/250,933,entitled “Displaying HDMI Content in a Tiled Window,” by Gaylord Yu,filed Aug. 30, 2016, which claims priority under 35 U.S.C. 119(e) toU.S. Provisional Application Ser. No. 62/211,825, “Displaying HDMIContent in a Tiled Window,” filed on Aug. 30, 2015, the contents of bothof which are herein incorporated by reference.

BACKGROUND

Field

The described embodiments relate to display techniques, includingdisplaying high-definition multimedia-interface (HDMI) content on adisplay in a tiled window.

Related Art

The versatility and capabilities of consumer-electronics orentertainment devices is increasing their popularity. For example, thecommunication capabilities of these entertainment devices allow users toaccess content from a wide variety of sources, including high-definitioncontent.

However, while the entertainment devices typically includehigh-resolution displays that allow users to view high-definitioncontent, the interface circuits and the communication bandwidth in manyentertainment devices can pose obstacles to simultaneous viewing ofhigh-definition content.

In addition, the user interfaces associated with many entertainmentdevices can be difficult to use. For example, the process of identifyingcontent from a particular source, selecting the content and having thecontent piped to a particular display often requires that users performmultiple operations. This convoluted process is time-consuming andcumbersome. Moreover, users often make mistakes when attempting tonavigate through a complicated set of options in different menus, whichfrustrates users and degrades their user experience.

SUMMARY

The described embodiments include an audio/video (A/V) hub. This A/V hubincludes: an antenna; an interface circuit that, during operation,communicates with a portable electronic device and an A/V displaydevice; and a control circuit coupled to the interface circuit. Duringoperation, the control circuit receives, via the interface circuit,user-interface activity information from the portable electronic devicethat specifies user selection of a content source. Then, the controlcircuit provides, via the interface circuit, a request forhigh-definition multimedia-interface (HDMI) content to the contentsource based on the user selection. Moreover, the control circuitreceives, via the interface circuit, the HDMI content from the contentsource. Next, the control circuit provides the HDMI content and displayinstructions to the A/V display device as frames with the HDMI contentare received from the content source, so that the HDMI content isdisplayed on a display in the A/V display device. Note that the displayinstructions specify that the HDMI content is to be displayed in a tiledwindow on the display while other HDMI content from another contentsource is displayed on the display.

The A/V hub may include a port that is coupled to the interface circuitand the A/V display device. This port may be compatible with an HDMIstandard.

Moreover, the user selection may involve activation of a virtual commandicon (such as activation of only a single virtual command icon).Alternatively, the user selection may involve activation of a physicalbutton (such as activation of only a single physical button). In someembodiments, during operation the control circuit: generatesuser-interface information that specifies a user interface that includesone or more virtual command icons, including the virtual command icon,which are associated with content sources; and provides theuser-interface information to the portable electronic device for displayon a touch-sensitive display in the portable electronic device.

The request from the A/V hub in response to the activation of thevirtual command icon or the physical button may include one or morecommands or instructions for the content source, including selectivelyactivating or turning on the content source (if the content source isnot already activated or turned on) and/or specifying the HDMI content.Therefore, in some embodiments, the request may be based, at least inpart, on device-state information about the content source.

Furthermore, the portable electronic devices may include: a remotecontrol for the A/V hub; and/or a cellular telephone.

Additionally, during operation the control circuit may determine thedisplay instructions based on a format of the display.

Note that providing the HDMI content and the display instructions mayinvolve transcoding the HDMI content based on a format of the display.

In some embodiments, during operation the control circuit: receives, viathe interface circuit, the other HDMI content from the other contentsource; and provides, via the interface circuit, the other HDMI contentto the A/V display device, so that the other HDMI content is displayedon the display. Moreover, during operation the control circuit mayprovide second display instructions that specify how the other HDMIcontent is to be displayed on the display. Furthermore, during operationthe control circuit may determine the second display instructions basedon a format of the display.

In some embodiments, the control circuit includes: a processor; and amemory, coupled to the processor, which stores a program module that,during operation, is executed by the processor. The program module mayinclude instructions for at least some operations performed by thecontrol circuit.

Another embodiment provides a computer-program product for use with theA/V hub. This computer-program product includes instructions for atleast some of the operations performed by the A/V hub.

Another embodiment provides a method for providing HDMI content. Thismethod includes at least some of the operations performed by the A/Vhub.

Another embodiment provides the A/V display device.

Another embodiment provides the portable electronic device.

This Summary is provided merely for purposes of illustrating someexemplary embodiments, so as to provide a basic understanding of someaspects of the subject matter described herein. Accordingly, it will beappreciated that the above-described features are merely examples andshould not be construed to narrow the scope or spirit of the subjectmatter described herein in any way. Other features, aspects, andadvantages of the subject matter described herein will become apparentfrom the following Detailed Description, Figures, and Claims.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 is a block diagram illustrating a system with electronic deviceswirelessly communicating in accordance with an embodiment of the presentdisclosure.

FIG. 2 is a flow diagram illustrating a method for providinghigh-definition multimedia-interface (HDMI) content in accordance withan embodiment of the present disclosure.

FIG. 3 is a drawing illustrating communication among the electronicdevices in FIG. 1 in accordance with an embodiment of the presentdisclosure.

FIG. 4 is a flow diagram illustrating a method for providing HDMIcontent in accordance with an embodiment of the present disclosure.

FIG. 5 is a drawing illustrating communication among the electronicdevices in FIG. 1 in accordance with an embodiment of the presentdisclosure.

FIG. 6 is a drawing illustrating a user interface in a portableelectronic device of FIG. 1 in accordance with an embodiment of thepresent disclosure.

FIG. 7 is a drawing illustrating a display in the audio/video (A/V)display device of FIG. 1 in accordance with an embodiment of the presentdisclosure.

FIG. 8 is a drawing illustrating a display in the A/V display device ofFIG. 1 in accordance with an embodiment of the present disclosure.

FIG. 9 is a drawing illustrating a display in the A/V display device ofFIG. 1 in accordance with an embodiment of the present disclosure.

FIG. 10 is a drawing illustrating a display in the A/V display device ofFIG. 1 in accordance with an embodiment of the present disclosure.

FIG. 11 is a drawing illustrating a display in the A/V display device ofFIG. 1 in accordance with an embodiment of the present disclosure.

FIG. 12 is a drawing illustrating a display in the A/V display device ofFIG. 1 in accordance with an embodiment of the present disclosure.

FIG. 13 is a drawing illustrating a display in the A/V display device ofFIG. 1 in accordance with an embodiment of the present disclosure.

FIG. 14 is a flow diagram illustrating a method for providing displayinstructions in accordance with an embodiment of the present disclosure.

FIG. 15 is a drawing illustrating communication among the electronicdevices in FIG. 1 in accordance with an embodiment of the presentdisclosure.

FIG. 16 is a drawing illustrating a display in the A/V display device ofFIG. 1 in accordance with an embodiment of the present disclosure.

FIG. 17 is a drawing illustrating a display in the A/V display device ofFIG. 1 in accordance with an embodiment of the present disclosure.

FIG. 18 is a flow diagram illustrating a method for casting of HDMIcontent in accordance with an embodiment of the present disclosure.

FIG. 19 is a drawing illustrating communication among the electronicdevices in FIG. 1 in accordance with an embodiment of the presentdisclosure.

FIG. 20 is a drawing illustrating a display in the A/V display device ofFIG. 1 in accordance with an embodiment of the present disclosure.

FIG. 21 is a drawing illustrating an A/V hub in FIG. 1 in accordancewith an embodiment of the present disclosure.

FIG. 22 is a block diagram illustrating a state-detection circuit in oneof the electronic devices of FIG. 1 in accordance with an embodiment ofthe present disclosure.

FIG. 23 is a flow diagram illustrating a method for detecting anentertainment device in accordance with an embodiment of the presentdisclosure.

FIG. 24 is a block diagram illustrating one of the electronic devices ofFIG. 1 in accordance with an embodiment of the present disclosure.

Note that like reference numerals refer to corresponding partsthroughout the drawings. Moreover, multiple instances of the same partare designated by a common prefix separated from an instance number by adash.

DETAILED DESCRIPTION

In a first group of embodiments, in response to receiving user-interfaceactivity information from a portable electronic device (such as acellular telephone or a remote control) that specifies user selection ofa content source, an audio/video (A/V) hub provides a request forhigh-definition multimedia-interface (HDMI) content to the contentsource based on the user selection. When the A/V hub receives the HDMIcontent from the content source, the A/V hub provide the HDMI contentand display instructions to an A/V display device (such as aconsumer-electronic device, e.g., a television) as frames with the HDMIcontent are received from the content source, so that the HDMI contentis displayed in a tiled window on the display in the A/V display devicewhile other HDMI content from another content source is displayed on thedisplay.

In a second group of embodiments, an A/V hub determines displayinstructions specifying information to be displayed on a display in anA/V display device that includes icons associated with content sources.These content sources are located at arbitrary locations in a structure,including locations external to an environment that includes the A/V huband the A/V display device. Then, the A/V hub provides the displayinstructions to the A/V display device for display on the display.

In a third group of embodiments, an A/V hub determines displayinstructions specifying information to be displayed on a display in anA/V display device that includes icons associated with content sources.Then, the A/V hub provides the display instructions to the A/V displaydevice for display on the display. In response to receiving a userselection of a content source based on activation of a single commandfeature in a portable electronic device that is associated with thecontent source, the A/V hub provides a request for HDMI content to thecontent source. When the A/V hub receives the HDMI content from thecontent source, the A/V hub provide the HDMI content and second displayinstructions to an A/V display device as frames with the HDMI contentare received from the content source, so that the HDMI content isdisplayed in a tiled window on the display in the A/V display device.

In a fourth group of embodiments, in response to receivinguser-interface activity information from a portable electronic devicethat specifies user instruction to cast HDMI content associated with acontent source from a first A/V display device to a second A/V displaydevice, an A/V hub accesses device-state information that specifies acurrent state of the second A/V display device. Then, the A/V hubperforms the casting of the HDMI content from the first A/V displaydevice to the second A/V display device using a dynamic number ofoperations based on the device-state information, so that the HDMIcontent is displayed on a display in the second A/V display device.

By dynamically adapting the displayed HDMI content in response to userselections, this display technique may make it easier for a user tocontrol an entertainment device (such as the A/V hub, the A/V displaydevice and/or one or more content sources) with fewer errors ormistakes. Moreover, by providing a simple and intuitive user interfacein the portable electronic device and display instructions for the A/Vdisplay device, this display technique may make it easier for a user to:select HDMI content from different content sources, change how the HDMIcontent is displayed and/or where the HDMI content is displayed.Consequently, the display technique may reduce user frustration and/ormay improve the user experience when using the portable electronicdevice, the A/V hub, the A/V display device and/or one or more contentsources.

In the discussion that follows the portable electronic device, the A/Vhub, the A/V display device, the one or more content sources, and/or anentertainment device (such as a consumer-electronic device) may includeradios that communicate packets or frames in accordance with one or morecommunication protocols, such as: an Institute of Electrical andElectronics Engineers (IEEE) 802.11 standard (which is sometimesreferred to as ‘Wi-Fi®,’ from the Wi-Fi® Alliance of Austin, Tex.),Bluetooth® (from the Bluetooth Special Interest Group of Kirkland,Wash.), a cellular-telephone communication protocol, anear-field-communication standard or specification (from the NFC Forumof Wakefield, Mass.), and/or another type of wireless interface. In thediscussion that follows, Wi-Fi is used as an illustrative example. Forexample, the cellular-telephone communication protocol may include ormay be compatible with: a 2^(nd) generation of mobile telecommunicationtechnology, a 3^(rd) generation of mobile telecommunications technology(such as a communication protocol that complies with the InternationalMobile Telecommunications-2000 specifications by the InternationalTelecommunication Union of Geneva, Switzerland), a 4^(th) generation ofmobile telecommunications technology (such as a communication protocolthat complies with the International Mobile Telecommunications Advancedspecification by the International Telecommunication Union of Geneva,Switzerland), and/or another cellular-telephone communication technique.In some embodiments, the communication protocol includes Long TermEvolution or LTE. However, a wide variety of communication protocols maybe used. In addition, the communication may occur via a wide variety offrequency bands. Note that the portable electronic device, the A/V hub,the A/V display device, the one or more content sources, and/or anentertainment device may communicate using infra-red communication thatis compatible with an infra-red communication standard (includingunidirectional or bidirectional infra-red communication).

Communication among electronic devices is shown in FIG. 1, whichpresents a block diagram illustrating a system 100 with a portableelectronic device 110 (such as a remote control or a cellulartelephone), an A/V hub 112, one or more A/V display devices 114 (such asa television, a monitor, a computer and, more generally, a displayassociated with an electronic device) and one or more content sources116 (e.g., a radio receiver, a video player, a satellite receiver, anaccess point that provides a connection to a wired network such as theInternet, a media or a content source, a consumer-electronic device, anentertainment device, a set-top box, over-the-top content delivered overthe Internet or a network without involvement of a cable, satellite ormultiple-system operator, etc.). (Note that A/V hub 112, the one or moreA/V display devices 114, and the one or more content sources 116 aresometimes collectively referred to as ‘components’ in system 100.However, A/V hub 112, the one or more A/V display devices 114, and theone or more content sources 116 are sometimes referred to asentertainment devices.) In particular, portable electronic device 110and A/V hub 112 may communicate with each other using wirelesscommunication, and A/V hub 112 and other components in system 100 (suchas the one or more A/V display devices 114 and the one or more contentsources 116) may communicate using wireless and/or wired communication.During the wireless communication, these electronic devices maywirelessly communicate while: transmitting advertising frames onwireless channels, detecting one another by scanning wireless channels,establishing connections (for example, by transmitting associationrequests), and/or transmitting and receiving packets or frames (whichmay include the association requests and/or additional information aspayloads, such as user-interface information, device-state information,user-interface activity information, data, A/V content, etc.).

As described further below with reference to FIG. 24, portableelectronic device 110, A/V hub 112, the one or more A/V display devices114 and the one or more content sources 116 may include subsystems, suchas: a networking subsystem, a memory subsystem and a processorsubsystem. In addition, portable electronic device 110 and A/V hub 112,and optionally one or more of the one or more A/V display devices 114and/or the one or more content sources 116, may include radios 118 inthe networking subsystems. (Note that radios 118 may be instances of thesame radio or may be different from each other.) More generally,portable electronic device 110 and A/V hub 112 (and optionally one ormore of the one or more A/V display devices 114 and/or the one or morecontent sources 116) can include (or can be included within) anyelectronic devices with the networking subsystems that enable portableelectronic device 110 and A/V hub 112 (and optionally one or more of A/Vdisplay devices 114 and/or the one or more content sources 116) towirelessly communicate with each other. This wireless communication cancomprise transmitting advertisements on wireless channels to enableelectronic devices to make initial contact or detect each other,followed by exchanging subsequent data/management frames (such asassociation requests and responses) to establish a connection, configuresecurity options (e.g., Internet Protocol Security), transmit andreceive packets or frames via the connection, etc.

As can be seen in FIG. 1, wireless signals 120 (represented by a jaggedline) are transmitted from radio 118-1 in portable electronic device110. These wireless signals are received by at least A/V hub 112. Inparticular, portable electronic device 110 may transmit packets. Inturn, these packets may be received by a radio 118-2 in A/V hub 112.This may allow portable electronic device 110 to communicate informationto A/V hub 112. While FIG. 1 illustrates portable electronic device 110transmitting packets, note that portable electronic device 110 may alsoreceive packets from A/V hub 112.

In the described embodiments, processing of a packet or frame inportable electronic device 110 and A/V hub 112 (and optionally one ormore of the one or more A/V display devices 114 and/or the one or morecontent sources 116) includes: receiving wireless signals 120 with thepacket or frame; decoding/extracting the packet or frame from receivedwireless signals 120 to acquire the packet or frame; and processing thepacket or frame to determine information contained in the packet orframe (such as the information associated with a data stream). Forexample, the information from portable electronic device 110 may includeuser-interface activity information associated with a user interfacedisplayed on touch-sensitive display 124 in portable electronic device110, which a user of portable electronic device 110 uses to control A/Vhub 112, the one or more A/V display devices 114 and/or one of the oneor more content sources 116. (In some embodiments, instead of or inadditional to touch-sensitive display 124, portable electronic device110 includes a use interface with physical knobs and/or buttons that auser can use to control A/V hub 112, the one or more A/V display devices114 and/or one of the one or more content sources 116.) Alternatively,the information from A/V hub 112 may include device-state informationabout a current device state of one or more of A/V display devices 114or one of the one or more content sources 116 (such as on, off, play,rewind, fast forward, a selected channel, selected content, a contentsource, etc.), or may include user-interface information for the userinterface (which may be dynamically updated based on the device-stateinformation and/or the user-interface activity information).Furthermore, the information from A/V hub 112 and/or one of the one ormore content sources 116 may include audio and video that are displayedon one or more of A/V display devices 114, as well as displayinstructions that specify how the audio and video are to be displayed.(However, as noted previously, the audio and video may be communicatedbetween components in system 100 via wired communication. Therefore, asshown in FIG. 1, there may be a wired cable or link, such as ahigh-definition multimedia-interface (HDMI) cable 122, between A/V hub112 and A/V display device 114-1.)

Note that the communication between portable electronic device 110 andA/V hub 112 (and optionally one or more of the one or more A/V displaydevices 114 and/or the one or more content sources 116) may becharacterized by a variety of performance metrics, such as: a data rate,a data rate for successful communication (which is sometimes referred toas a ‘throughput’), an error rate (such as a packet error rate, or aretry or resend rate), a mean-square error of equalized signals relativeto an equalization target, intersymbol interference, multipathinterference, a signal-to-noise ratio, a width of an eye pattern, aratio of number of bytes successfully communicated during a timeinterval (such as 1-10 s) to an estimated maximum number of bytes thatcan be communicated in the time interval (the latter of which issometimes referred to as the ‘capacity’ of a channel or link), and/or aratio of an actual data rate to an estimated data rate (which issometimes referred to as ‘utilization’). Moreover, the performanceduring the communication associated with different channels may bemonitored individually or jointly (e.g., to identify dropped packets).

The communication between portable electronic device 110 and A/V hub 112(and optionally one or more of the one or more A/V display devices 114and/or the one or more content sources 116) in FIG. 1 may involve one ormore independent, concurrent data streams in different wireless channels(or even different communication protocols, such as different Wi-Ficommunication protocols) in one or more connections or links, which maybe communicated using multiple radios. Note that the one or moreconnections or links may each have a separate or different service setidentifier on a wireless network in system 100 (which may be aproprietary network or a public network). Moreover, the one or moreconcurrent data streams may, on a dynamic or packet-by-packet basis, bepartially or completely redundant to improve or maintain the performancemetrics even when there are transient changes (such as interference,changes in the amount of information that needs to be communicated,movement of portable electronic device 110, etc.), and to facilitateservices (while remaining compatible with the communication protocol,e.g., a Wi-Fi communication protocol) such as: channel calibration,determining of one or more performance metrics, performingquality-of-service characterization without disrupting the communication(such as performing channel estimation, determining link quality,performing channel calibration and/or performing spectral analysisassociated with at least one channel), seamless handoff betweendifferent wireless channels, coordinated communication betweencomponents, etc. These features may reduce the number of packets thatare resent, and, thus, may decrease the latency and avoid disruption ofthe communication and may enhance the experience of one or more users orviewers of content on the one or more A/V display devices 114.

As noted previously (and described further below with reference to FIG.6), a user may control A/V hub 112, one or more of the one or more A/Vdisplay devices 114 and/or one of the one or more content sources 116via the user interface displayed on touch-sensitive display 124 onportable electronic device. In particular, at a given time, the userinterface may include one or more virtual icons that allow the user toactivate, deactivate or change functionality or capabilities of A/V hub112, one or more of A/V display devices 114 and/or one or more ofcontent sources 116. For example, a given virtual icon in the userinterface may have an associated strike area on a surface oftouch-sensitive display 124. If the user makes and then breaks contactwith the surface (e.g., using one or more fingers or digits, or using astylus) within the strike area, portable electronic device 110 (such asa processor executing a program module) may receive user-interfaceactivity information indicating activation of this command orinstruction from a touch-screen input/output (I/O) controller, which iscoupled to touch-sensitive display 124. (Alternatively, touch-sensitivedisplay 124 may be responsive to pressure. In these embodiments, theuser may maintain contact with touch-sensitive display 124 with anaverage contact pressure that is usually less than a threshold value,such as 10-20 kPa, and may activate a given virtual icon by increase theaverage contact pressure with touch-sensitive display 124 above thethreshold value.) In response, the program module may instruct aninterface circuit in portable electronic device 110 to wirelesslycommunicate the user-interface activity information indicating thecommand or instruction to A/V hub 112, and A/V hub 112 may communicatethe command or the instruction to the target component in system 100(such as A/V display device 114-1). This instruction or command mayresult in A/V display device 114-1 turning on or off, displaying contentfrom a particular source, performing a trick mode of operation (such asfast forward, reverse, fast reverse or skip), etc.

One problem with using existing remote controls to control the operationof another component or entertainment device is that the remote controldoes not receive any feedback from the entertainment device. Forexample, many existing remote controls use infra-red communication.However, typically existing infra-red communication protocols areunidirectional or one-way communication, i.e., from a remote control tothe entertainment device. Consequently, the remote control usually doesnot have any knowledge of the effects of the commands or instructionsthat are communicated to the entertainment device. In particular, theremote control is typically unaware of a current state of theentertainment device, such as whether the entertainment device is in: apower-on state, a power-off state, a playback state, a trick-mode state(such as fast forward, fast reverse, or skip), a pause state, a standby(reduced-power) state, a record state, a state in which contentassociated with a given content source (such as cable television, asatellite network, a web page on the Internet, etc.) is received orprovided, and/or another state. (Note that one or more of the states maybe nested or concurrent with each other, such as the power-on state andthe playback state.) By operating blindly in this way, existing remotecontrol are unable to leverage knowledge of the current state of theentertainment device to improve the user experience.

This problem is addressed in system 100. In particular, as describedfurther below with reference to FIGS. 22 and 23, A/V hub 112 maydetermine the current state of one or more of the components in system100, such as the current state of A/V display device 114-1 and/or one ofthe one or more content sources 116. This device-state information maybe determined by A/V hub 112 using hardware and/or software, and A/V hub112 may determine the device-state information even for legacyentertainment devices that are only capable of receiving commands orinstructions (i.e., that are only capable of unidirectionalcommunication). For example, as described further below with referenceto FIGS. 22 and 23, whether or not a given component or entertainmentdevice in system 100 is electrically coupled to A/V hub 112 may bedetermined using a state-detection circuit that detects whether there iselectrical coupling between the entertainment device and an inputconnector to A/V hub 112 (such as an HDMI connector or port that can beelectrically coupled to HDMI cable 122). If the electrical coupling isdetected, the type of the given entertainment device (such as atelevision, a DVD player, a satellite receiver, etc.) and/or themanufacturer or provider of the given entertainment device may bedetermined by A/V hub 112 by providing a series of commands orinstructions to the given entertainment device (e.g., such as commandsor instructions that are specific to a particular type of entertainmentdevice, specific to a particular manufacturer, and/orconsumer-electronics-control commands in the HDMI standard orspecification), and then monitoring, as a function of time, changes in adata stream (as indicated by the number of packets or frames and/or thepayloads in the packets or frames) to and/or from the givenentertainment device to see if there was a response to a particularcommand or instruction. Moreover, the state-detection circuit maydetermine whether the given entertainment device is in the power-onstate or the power-off state by monitoring a voltage, a current and/oran impedance on, through or associated with one or more pins in theinput connector. Alternatively or additionally, A/V hub 112 maydetermine whether the given entertainment device is in the power-onstate or the power-off state by monitoring, as a function of time, thedata stream (as indicated by the number of packets or frames and/or thepayloads in the packets or frames) to and/or from the givenentertainment device. Similarly, A/V hub 112 may determine the currentstate of the given entertainment device, such as whether the givenentertainment device responded to a command or instruction that wasprovided to the given entertainment device by A/V hub 112, bymonitoring, as a function of time, changes in the data stream (asindicated by the number of packets or frames and/or the payloads in thepackets or frames) to and/or from the given entertainment device. Thus,the device-state information for the given entertainment devicedetermined by A/V hub 112 may include: presence or absence information(such as whether there is electrical coupling or a wireless connectionwith the given entertainment device), identity information (such as thetype of the given entertainment device and/or the manufacturer of thegiven entertainment device) and/or the current state.

Using the device-state information A/V hub 112 and/or portableelectronic device 110 may dynamically adapt the user interface displayedon touch-sensitive display 124 on portable electronic device 110. Forexample, A/V hub 112 may provide, via radio 118-2, device-stateinformation to portable electronic device 110 specifying a current stateof the given entertainment device. (Thus, this feedback technique mayinclude bidirectional or two-way communication between A/V hub 112 andportable electronic device 110.) After radio 118-1 receives thedevice-state information, portable electronic device 110 (such as aprogram module executed in an environment, e.g., an operating system, inportable electronic device 110) may generate a user interface thatincludes one or more virtual command icons associated with the currentstate and one or more related states of the given entertainment device.Note that the one or more related states may be related to the currentstate in a state diagram (which may be stored in memory in portableelectronic device 110) by corresponding operations that transition thegiven entertainment device from the current state to the one or morerelated states. Then, portable electronic device 110 may display theuser interface on touch-sensitive display 124.

In some embodiments, A/V hub 112 provides information specifying thetype of the given entertainment device, the manufacturer of the givenentertainment device, and/or context information that specifies acontext of content (such as A/V content) displayed on the entertainmentdevice (such as A/V display device 114-1). For example, the context mayinclude a type of the content (such as sports, television, a movie,etc.), a location in the content (such as a timestamp, an identifier ofa sub-section in the content and/or a proximity to a beginning or an endof the content), etc. In these embodiments, the one or more virtualcommand icons (and, thus, the user interface) may be based on the typeof the given entertainment device, the manufacturer and/or the context.Thus, only virtual command icons that are relevant to the givenentertainment device, the manufacturer and/or the context may beincluded in the user interface.

Moreover, when the user activates one of the virtual command icons inthe user interface, the touch-screen I/O controller in portableelectronic device 110 may provide user-interface activity informationspecifying activation of a virtual command icon in the one or morevirtual command icons, where the activation of the virtual command iconspecifies a transition of the given entertainment device from thecurrent state to a new current state in the state diagram. As notedpreviously, the activation of the virtual command icon may involve auser of portable electronic device 110 contacting touch-sensitivedisplay 124 within a strike area of the virtual command icon and thenreleasing the contact. In response to receiving the user-interfaceactivity information, portable electronic device 110 may: modify theuser interface to change the one or more virtual command icons based onthe new current state; and display the modified user interface ontouch-sensitive display 124. Note that portable electronic device 110may wait to change the one or more virtual command icons until thedevice-state information received from A/V hub 112 indicates that thegiven entertainment device has transitioned to the new current state inresponse to a command or an instruction associated with the activationof the one of the virtual command icons. Thus, portable electronicdevice 110 may repeatedly perform the generating and the displayingoperations so that the user interface is dynamically updated as thecurrent state changes.

Alternatively or additionally, instead of portable electronic device 110generating the user interface, A/V hub 112 may generate user-interfaceinformation that specifies the user interface (or instructionsspecifying objects or graphical information in the user interface) basedon the one or more related states in the state diagram (which may bestored in memory in A/V hub 112) and one or more of: the device-stateinformation, the type of the given entertainment device, themanufacturer of the given entertainment device, the context,user-interface activity information specifying activation (by the user)of one of the virtual command icons in the user interface (which may bereceived, via radios 118, from portable electronic device 110), and/or adisplay format in portable electronic device 110. Then, A/V hub 112 mayprovide, via radios 118, the user-interface information to portableelectronic device 110 for display on touch-sensitive display 124.

In this way, the user interface may be dynamically updated as thecomponents in system 100 respond to commands or instructions receivedfrom portable electronic device 110 and/or A/V hub 112, so that thecurrently relevant one or more virtual icons are included in the userinterface. This capability may simplify the user interface and make iteasier for the user to navigate through and/or use the user interface.

Moreover, as described further below with reference to FIGS. 18 and 19,the device-state information may be used by A/V hub 112 to dynamicallyadapt or change a number of operations needed or used to cast HDMIcontent from one A/V display device to another A/V display device, suchas from A/V display device 114-1 to A/V display device 114-2 (which maybe in the same room or a different room in the environment that includesA/V hub 112). (While HDMI content is used as an illustrative example, inother embodiments content that is compatible with another format orstandard is used in the embodiments of the display technique, including:H.264, MPEG-2, a QuickTime video format, MPEG-4 and/or MP4. Moreover,the video mode of the content may be 720p, 1080i, 1080p, 1440p, 2000,2160p, 2540p, 4000p and/or 4320p) In particular, in response touser-interface activity information received from portable electronicdevice 110 that specifies user instruction to cast HDMI contentassociated with a content source (such as content source 116-1) from A/Vdisplay device 114-1 to A/V display device 114-2 (such as useractivation of a virtual command icon in a user interface), A/V hub 112may access device-state information that specifies a current state ofA/V display device 114-2. Then, A/V hub 112 may perform the casting ofthe HDMI content from A/V display device 114-1 to A/V display device114-2 (and, in particular, from the content source to A/V display device114-2) using a dynamic number of operations based on the device-stateinformation, so that the HDMI content is displayed on a display in A/Vdisplay device 114-2. For example, when the device-state informationindicates that A/V display device 114-2 is powered on, a power-onoperation may be excluded from the dynamic number of operations.Alternatively or additionally, when the device-state informationindicates that A/V display device 114-2 is set to receive the HDMIcontent from content source 116-1, an operation to set A/V displaydevice 114-2 to receive the HDMI content from content source 116-1 maybe excluded from the dynamic number of operations. Thus, based ondevice-state awareness, the display technique may adapt or change thedynamic number of operations so that operations such as casting areperformed efficiently (i.e., are less time-consuming).

Furthermore, as described further below with reference to FIGS. 2 and 3,in response to user-interface activity information received fromportable electronic device 110 that specifies a user selection of acontent source (such as content source 116-1), A/V hub 112 may provide arequest for HDMI content to content source 116-1 based on the userselection. When A/V hub 112 receives the HDMI content from contentsource 116-1, A/V hub 112 may provide the HDMI content and displayinstructions to A/V display device 114-1 as frames with the HDMI contentare received from content source 116-1 (e.g., in real time), so that theHDMI content is displayed in a tiled window on the display in A/Vdisplay device 114-1 while other HDMI content from another contentsource (such as content source 116-2) is displayed on the display. Notethat a ‘tiled window’ may include window in at least a portion of thedisplay area that does not overlap another window, but that presents theHDMI content overlaid on a background of the window. (Alternatively oradditionally, the HDMI content may be displayed in a cascaded oroverlapped window that at least partially overlaps or that is partiallyoverlapped by another window.) Thus, the display technique may allowHDMI content to be presented in a tiled window in real time (such asframes with the HDMI content are received from content source 116-1).

In some embodiments, as described further below with reference to FIGS.4 and 5, A/V hub 112 determines display instructions specifyinginformation to be displayed on the display in an A/V display device(such as A/V display device 114-1) that includes icons associated withcontent sources (such as content sources 116). Then, A/V hub 112provides the display instructions to A/V display device 114-1 fordisplay on the display. For example, the information to be displayed mayspecify content (such as HDMI content) that is displayed on the displayand where the content is displayed (such as one or more tiled windows).Therefore, the information to be displayed may be based on a format ofthe display, such as: a display size, display resolution, display aspectratio, display contrast ratio, a display type, etc. Then, in response toreceiving a user selection of content source 116-1 (or user-interfaceactivity information that specifies the user selection) based onactivation of a single command feature in portable electronic device 110that is associated with content source 116-1 (such as a virtual commandicon in a user interface displayed on portable electronic device 110),A/V hub 112 may provide a request for HDMI content to content source116-1. When A/V hub 112 receives the HDMI content from content source116-1, A/V hub 112 may provide the HDMI content and second displayinstructions to A/V display device 114-1 as frames with the HDMI contentare received from content source 116-1, so that the HDMI content isdisplayed in a tiled window on the display in A/V display device 114-1.For example, the information to be displayed may specify that the HDMIcontent is displayed on the display in a new tiled window. Thus, thedisplay technique may allow the new tiled window to be created in thedisplay based on selection or activation of the single command featurewithout requiring further action or additional operations be performedby the user.

Additionally, as described further below with reference to FIGS. 14-17,A/V hub 112 may determine the display instructions specifying theinformation to be displayed on the display in an A/V display device(such as A/V display device 114-1) that includes icons associated withcontent sources 116. These content sources may be located at arbitrarylocations in a structure (such as one or more rooms in a building),including locations external to an environment (such as at least aportion of a room in the building, which may be different than the oneor more rooms) that includes A/V hub 112 and A/V display device 114-1.Then, A/V hub 112 provides the display instructions to A/V displaydevice 114-1 for display on the display. Thus, leveraging the wiredand/or wireless network(s) associated with A/V hub 112, the displaytechnique may allow a wide variety of content sources provided bydifferent providers, and which are locally or remoted located, to bedisplayed on A/V display device 114-1.

In these ways, the display technique may allow a user to control the A/Vhub, one or more A/V display devices and/or one or more content sourceswith less effort (such as in less time, with fewer operations and/orwith less confusion), and to do so with fewer errors or mistakes.Consequently, the display technique may reduce user frustration and/ormay improve the user experience when using the portable electronicdevice, the A/V hub, the one or more A/V display devices and/or the oneor more content sources.

Although we describe the network environment shown in FIG. 1 as anexample, in alternative embodiments, different numbers or types ofelectronic devices may be present. For example, some embodimentscomprise more or fewer electronic devices. As another example, inanother embodiment, different electronic devices are transmitting and/orreceiving packets or frames. While portable electronic device 110 andA/V hub 112 are illustrated with a single instance of radios 118, inother embodiments portable electronic device 110 and A/V hub 112 (andoptionally one or more of A/V display device 114-1 and/or the one ormore content sources 116) may include multiple radios.

We now describe embodiments of a display technique. FIG. 2 presents aflow diagram illustrating method 200 for providing HDMI content, whichmay be performed by an A/V hub, such as A/V hub 112 (FIG. 1). Duringoperation, the A/V hub (such as a control circuit or control logic,e.g., a processor executing a program module and/or or a circuit)receives, via an interface circuit in the A/V hub, user-interfaceactivity information (operation 210) from a portable electronic device,where the user-interface activity information specifies user selectionof a content source. For example, the user selection may involveactivation of a virtual command icon associated with the content source,which is displayed in a user interface on the portable electronicdevice. Alternatively, the user selection may involve activation of aphysical button associated with the content source, which is included inthe portable electronic device. Note that the A/V hub may optionally:generate user-interface information that specifies the user interfacethat includes one or more virtual command icons, including the virtualcommand icon, which are associated with one or more content sources; andprovides the user-interface information to the portable electronicdevice for display on a touch-sensitive display in the portableelectronic device. However, in other embodiments the user-interfaceinformation is optionally generated by the portable electronic device.

Then, the A/V hub provides, via the interface circuit, a request forhigh-definition multimedia-interface (HDMI) content (operation 212) tothe content source based on the user selection. Moreover, the A/V hubreceives, via the interface circuit, the HDMI content (operation 214)from the content source, and the A/V hub may optionally determine thedisplay instructions (operation 216) based on a format of the display.The request from the A/V hub in response to the activation of thevirtual command icon or the physical button may include one or morecommands or instructions for the content source, including selectivelyactivating or turning on the content source (if the content source isnot already activated or turned on) and/or specifying the HDMI content.Therefore, in some embodiments, the request may be based, at least inpart, on device-state information about the content source.

Next, the A/V hub provides, via the interface circuit, the HDMI contentand display instructions (operation 218) to an A/V display device asframes with the HDMI content are received from the content source, sothat the HDMI content is displayed on a display in the A/V displaydevice. For example, providing the HDMI content and the displayinstructions may involve transcoding the HDMI content based on a formatof the display. Note that the display instructions specify that the HDMIcontent is to be displayed in a tiled window on the display while otherHDMI content from another content source is displayed on the display.

In some embodiments, the A/V hub optionally performs one or moreadditional operations (operation 220). For example, the A/V hub mayoptionally: receive, via the interface circuit, the other HDMI contentfrom the other content source; and provide, via the interface circuit,the other HDMI content to the A/V display device, so that the other HDMIcontent is displayed on the display. Moreover, the A/V hub mayoptionally provide second display instructions that specify how theother HDMI content is to be displayed on the display. Furthermore, theA/V hub may optionally determine the second display instructions basedon a format of the display.

Note that method 200 may address security issues associated with theconcurrent display of the tiled window(s). For example, the HDMI contentmay be displayed in the tiled window on the display while the other HDMIcontent from the other content source is displayed on the displaywithout a risk of hacking (such as intentional communication ofmalicious content, such as malicious software, malware or a so-called‘virus’).

FIG. 3 presents a drawing illustrating communication among theelectronic devices in FIG. 1, which presents a drawing illustratingcommunication between portable electronic device 110, A/V hub 112 andA/V display device 114-1. In particular, processor 310 may optionallygenerate user-interface information 312 that specifies a user interfacethat includes one or more virtual command icons, including a virtualcommand icon, which are associated with content sources 116. Then,processor 310 may optionally provide, via interface circuit 314,user-interface information 312 to portable electronic device 110. Afterreceiving user-interface information 312, portable electronic device 110may optionally display associated user interface 316 on atouch-sensitive display (such as touch-sensitive display 124 in FIG. 1)in portable electronic device 110. However, in other embodimentsuser-interface information 312 is optionally generated by portableelectronic device 110.

For example, interface circuit 318 in portable electronic device 110 mayreceive user-interface information 312, which is then provided toprocessor 320. Alternatively, processor 320 may generate user-interfaceinformation 312. Then, based on user-interface information 312,processor 320 may provide information specifying user interface 316 totouch-sensitive input/output (I/O) controller 322, which provides theinformation specifying user interface 316 to touch-sensitive display124.

Moreover, touch-sensitive display 124 may provide to informationspecifying user interaction 324 to touch-sensitive I/O controller 322.In turn, touch-sensitive I/O controller 322 may interpret theinformation specifying user interaction 324 to determine user-interfaceactivity information 326. For example, user-interface activityinformation 326 may specify user selection of one of content sources116, such as user activation of the virtual command icon associated withthe one of content sources 116. Touch-sensitive I/O controller 322 mayprovide user-interface activity information 326 to processor 320, whichmay provide user-interface activity information 326 to interface circuit318.

Next, portable electronic device 110 (such as via interface circuit 318)may provide user-interface activity information 326 to A/V hub 112.After receiving user-interface activity information 326, interfacecircuit 314 may provide user-interface activity information 326 toprocessor 310. In response, processor 310 may instruct interface circuit314 to provide request 328 for HDMI content 330 to the one of contentsources 116. In addition, processor 310 may optionally determine displayinstructions 332 based on a format of a display in A/V display device114-1. Alternatively, display instructions 332 may be predetermined orpredefined.

After receiving request 328, the one of content sources 116 may provideHDMI content 330 to A/V hub 112. Next, interface circuit 314 may provideHDMI content 330 and/or display instructions 332 (which may be providedifferentially when there or changes or regularly, such in each packetor in one of every N packets) to A/V display device 114-1 as frames withHDMI content 330 are received from the one of content sources 116, sothat HDMI content 330 is displayed on the display in A/V display device114-1. (Alternatively, in some embodiments interface circuit 314provides HDMI content 330 to processor 310, which instructs interfacecircuit 314 to provide HDMI content 330 and display instructions 332 toA/V display device 114-1 as frames with HDMI content 330 are receivedfrom the one of content sources 116.)

Note that display instructions 332 may specify that HDMI content 330 isto be displayed in a tiled window on the display while additional HDMIcontent from another of content sources 116 is displayed on the display.For example, A/V hub 112 may optionally: receives, via interface circuit314, the additional HDMI content from the other of content sources 116;and provides, via interface circuit 314, the additional HDMI content toA/V display device 114-1, so that additional HDMI content is displayedon the display. Moreover, processor 310 may optionally determineadditional display instructions that specify how the additional HDMIcontent is to be displayed on the display based on a format of thedisplay. Furthermore, interface circuit 314 may optionally provide theadditional display instructions to A/V display device 114-1.

The user may create, modify or discontinue a tiled window by activatinga single command feature in a user interface displayed on the portableelectronic device. This is shown in FIG. 4, which presents a flowdiagram illustrating a method 400 for providing HDMI content. Note thatthis method may be performed by an A/V hub, such as A/V hub 112 (FIG.1). During operation, the A/V hub (such as a control circuit or controllogic, e.g., a processor executing a program module and/or or a circuit)determines display instructions (operation 410) specifying informationto be displayed on a display in an A/V display device, where theinformation to be displayed includes icons associated with contentsources. Then, the A/V hub provides, via an interface circuit in the A/Vhub, the display instructions (operation 412) to the A/V display devicefor display on the display. Moreover, the A/V hub receives, via theinterface circuit, user-interface activity information (operation 414)from a portable electronic device, where the user-interface activityinformation specifies user selection of a content source based onactivation of a single command feature in the portable electronic devicethat is associated with the content source. For example, the singlecommand feature may include a virtual command icon displayed in the userinterface on a touch-sensitive display in the portable electronicdevice. Alternatively, the single command feature may include a physicalbutton. Note that the A/V hub may optionally: generate user-interfaceinformation that specifies the user interface that includes one or morevirtual command icons, including the virtual command icon, which areassociated with the content sources; and provides the user-interfaceinformation to the portable electronic device for display on thetouch-sensitive display in the portable electronic device. However, inother embodiments the user-interface information is optionally generatedby the portable electronic device.

Next, the A/V hub provides, via the interface circuit, a request forHDMI content (operation 416) to the content source based on the userselection, and receives, via the interface circuit, the HDMI content(operation 418) from the content source. In some embodiments, the A/Vhub optionally determines second display instructions (operation 420)based on a format of the display. The request from the A/V hub inresponse to the activation of the virtual command icon or the physicalbutton may include one or more commands or instructions for the contentsource, including selectively activating or turning on the contentsource (if the content source is not already activated or turned on)and/or specifying the HDMI content. Therefore, in some embodiments, therequest may be based, at least in part, on device-state informationabout the content source.

Furthermore, the A/V hub provides the HDMI content and the seconddisplay instructions (operation 422) to the A/V display device as frameswith the HDMI content are received from the content source, so that theHDMI content is displayed on a display in the A/V display device, wherethe second display instructions specify that the HDMI content is to bedisplayed in a tiled window on the display. For example, providing theHDMI content and the second display instructions may involve transcodingthe HDMI content based on a format of the display.

Additionally, the A/V hub may optionally perform one or more additionaloperations (operation 424). For example, the A/V hub may: receive, viathe interface circuit, second user-interface activity information fromthe portable electronic device, where the second user-interface activityinformation specifies user de-activation of the content source based onactivation of the single command feature in the portable electronicdevice that is associated with the content source; provide, via theinterface circuit, a request to discontinue the HDMI content to thecontent source; and provide third display instructions to the A/Vdisplay device, so that display of the tiled window on the display isdiscontinued.

Alternatively or additionally, the A/V hub may: receive, via theinterface circuit, third user-interface activity information from theportable electronic device, where the third user-interface activityinformation specifies user selection of a second content source based onactivation of a second single command feature in the portable electronicdevice that is associated with the second content source; provide, viathe interface circuit, a request for second HDMI content to the secondcontent source; receive, via the interface circuit, the second HDMIcontent from the second content source; and provide the second HDMIcontent and third display instructions to the A/V display device asframes with the second HDMI content are received from the second contentsource, so that the second HDMI content is displayed on the display inthe A/V display device concurrently with the HDMI content in the titledwindow. Note that the third display instructions specify that the secondHDMI content is to be displayed in a second tiled window on the display.

Furthermore, subsequently, the A/V hub may receive, via the interfacecircuit, fourth user-interface activity information from the portableelectronic device that specifies user selection of one of the tiltedwindow and the second tiled window. In response, the A/V hub may providefourth display instructions to the A/V display device, so that givenHDMI content, which is one of the HDMI content and the second HDMIcontent, is displayed in a central tiled window on the display anddisplay of a given tiled window, which is one of the tiled window andthe second tiled window, is discontinued.

FIG. 5 presents a drawing illustrating communication among theelectronic devices in FIG. 1, which presents a drawing illustratingcommunication between portable electronic device 110, A/V hub 112, A/Vdisplay device 114-1 and one or more content sources 116. In particular,processor 310 may determine display instructions 510 specifyinginformation to be displayed on a display in an A/V display device 114-1.Then, processor 310 may optionally provide, via interface circuit 314,display instructions 510 to A/V display device 114-1 for display on thedisplay.

Moreover, processor 310 may optionally provide, via interface circuit314, user-interface information 512 to portable electronic device 110.After receiving user-interface information 512, portable electronicdevice 110 may optionally display an associated user interface on atouch-sensitive display (such as touch-sensitive display 124 in FIG. 1)in portable electronic device 110. However, in other embodimentsuser-interface information 512 is optionally generated by portableelectronic device 110.

Next, portable electronic device 110 may provide user-interface activityinformation 514 that specifies user selection of one of content sources116 (such as user activation of a single command feature associated withthe one of content sources 116) to A/V hub 112. After receivinguser-interface activity information 514, interface circuit 314 mayprovide user-interface activity information 514 to processor 310. Inresponse, processor 310 may instruct interface circuit 314 to providerequest 516 for HDMI content 518 to the one of content sources 116. Inaddition, processor 310 may optionally determine display instructions520 based on a format of the display in A/V display device 114-1.Alternatively, display instructions 520 may be predetermined orpredefined.

After receiving request 516, the one of content sources 116 may provideHDMI content 518 to A/V hub 112. Furthermore, interface circuit 314 mayprovide HDMI content 518 and display instructions 520 to A/V displaydevice 114-1 as frames with HDMI content 518 are received from the oneof content sources 116, so that HDMI content 518 is displayed on thedisplay in A/V display device 114-1. Note that display instructions 520may specify that HDMI content 518 is to be displayed in a tiled windowon the display while additional HDMI content from another of contentsources 116 is displayed on the display. (Alternatively, in someembodiments interface circuit 314 provides HDMI content 518 to processor310, which instructs interface circuit 314 to provide HDMI content 518and display instructions 1520 to A/V display device 114-1 as frames withHDMI content 518 are received from the one of content sources 116.)

In response to receiving, via interface circuit 314, user-interfaceactivity information 522 from portable electronic device 110, processor310 may optionally determine and provide display instructions 524 to A/Vdisplay device 114-1 and/or optional command 526 to the one of contentsources 116. For example, if user-interface activity information 522specifies de-activation of the one of content sources 116 based on useractivation of the single command feature (or another single commandfeature) in portable electronic device 110, display instructions 524 mayindicate that the tiled window on the display is discontinued. Inaddition, processor 310 may provide optional command 526 to the one ofcontent sources 116 to discontinue HDMI content 518. Alternatively,user-interface activity information 522 may specify user selection ofthe tilted window, and display instructions 524 may indicate that HDMIcontent 518 is displayed in a central tiled window on the display anddisplay of the tiled window on the display is discontinued.

In some embodiments, user-interface activity information 522 specifiesuser selection of another one of content sources 116 (such as useractivation of another single command feature associated with the otherone of content sources 116). In these embodiments, optional command 526requests additional HDMI content from the other one of content sources116. As the additional HDMI content is received by A/V hub 112,interface circuit 314 may provide the additional HDMI content anddisplay instructions 524 to A/V display device 114-1 as frames with theadditional HDMI content are received from the one of content sources116, so that the additional HDMI content is displayed on the display inA/V display device 114-1. Note that display instructions 524 may specifythat the additional HDMI content is to be displayed in another tiledwindow on the display while HDMI content 518 from the one of contentsources 116 is displayed on the display in the tiled window.

In this way, the display technique may make it easier and more intuitivefor a user to control the content and the layout of the contentdisplayed on the A/V display device (such as how and where the contentis displayed). In the process, the display technique may reduce userfrustration, and thus may improve user satisfaction when using theportable electronic device, the A/V hub, the A/V display device and/orthe one or more content sources.

In an exemplary embodiment, the portable electronic device includes atouch-sensitive display and the user interface includes features (suchas virtual command icons) associated with the content sources and thelayout of the content displayed on the A/V display device. This userinterface is shown in FIG. 6, which presents a drawing illustrating auser interface 600 on a touch-sensitive display 124 in portableelectronic device 110 in FIG. 1. In particular, user interface 600 mayinclude one or more virtual command icons 610 that are associated withcontent sources (such as content sources 116 in FIG. 1). By activatingone of virtual command icons 610 (such as by using a finger or digit totouch and then break contact with a surface of the touch-sensitivedisplay within a strike area associated with the virtual command icon),a user may provide user-interface activity information that instructs anA/V hub (such as A/V hub 112 in FIG. 1) to request HDMI content from theone of the content sources (such as the DVR) and, after receiving theHDMI content, to provide it along with display instructions to an A/Vdisplay device (such as one of A/V display devices 114 in FIG. 1). Asnoted previously, the HDMI content may be presented on a display in theA/V display device in a tiled window in real-time (i.e., as frames ofthe HDMI content are received from the content source). Note that theA/V hub may collect the HDMI content in a buffer until a frame isreceived, and then the A/V hub may provide the complete frame to the A/Vdisplay device. Alternatively, the A/V hub may provide packets withportions of a frame to the A/V display device as they are received.

In some embodiments, if the user makes and maintains contact with thesurface of the touch-sensitive display within a strike area associatedwith one of virtual command icons 610 or if the user activates thisvirtual command icon, the resulting user-interface activity informationfrom the portable electronic device instructs the A/V hub to providedisplay instructions for a list of display options to be displayed onthe A/V display device. In addition, if the user makes and maintainscontact with the surface of the touch-sensitive display within a strikearea associated with one of virtual command icons 610 or if the useractivates this virtual command icon, an associated one of virtualcommand icons 612 may be displayed. In particular, user interface 600may be modified (either by the portable electronic device and/or inresponse to user-interface information received from the A/V hub) toinclude one of virtual command icons 612 associated with a displayoption. For example, the display option may include: discontinuing thedisplay of the HDMI content from the content source (and, thus, todiscontinue or close the tiled window), as indicated by the circle withthe ‘X’ for virtual command icon 612-1; and/or, if the one of virtualcommand icons 612 associated with a new content source (i.e., a contentsource whose HDMI content is not currently displayed), displaying theHDMI content from the new content source in another tiled window, asindicated by the circles with the ‘+’ for virtual command icon 612-2. Inparticular, if the user makes and maintains contact with the surface ofthe touch-sensitive display within a strike area associated with virtualcommand icon 610-1 or if the user activates this virtual command icon,virtual command icon 612-1 may be displayed. Thus, if the user touchesvirtual command icon 610-1, the corresponding one of virtual commandicons 612 (the circle with the ‘X’) may be displayed. Then, if the userslides the contact area over virtual command icon 612-1 and releases thecontact (thereby providing an activation command), the portableelectronic device providing user-interface activation information to theA/V hub that instructs the A/V hub to communicate with the DVR todiscontinue the data stream with content (such as HDMI content) from theDVR. Alternatively, if the user makes and maintains contact with thesurface of the touch-sensitive display within a strike area associatedwith virtual command icon 610-2 or if the user activates this virtualcommand icon, virtual command icon 612-2 may be displayed. Thus, if theuser touches virtual command icon 610-2, the corresponding one ofvirtual command icons 612 (the circle with the ‘+’) may be displayed.Then, if the user slides the contact area over virtual command icon612-2 and releases the contact (thereby providing an activationcommand), the portable electronic device providing user-interfaceactivation information to the A/V hub that instructs the A/V hub tocommunicate with the Blue-Ray player to initiate a data stream withcontent (such as HDMI content) that will be displayed on the A/V displaydevice (such as in a tiled window). Note, therefore, that virtualcommand icons 612 may be dynamically displayed in the user interfacedepending on which virtual command icon 610 is selected, and may dependupon the current state of the associated content source, as well aswhether at least one other content source is currently providing a datastream with content (such as HDMI content).

Note that the content sources may be associated with different providersand may be at arbitrary locations in an environment (such as indifferent rooms in a structure than the A/V hub and the A/V displaydevice). Thus, the display technique may be independent of orinsensitive to the actual location of the content source(s) that providethe HDMI content. In these embodiments, if the user makes and maintainscontact with the surface of the touch-sensitive display within a strikearea associated with one of virtual command icons 610 and/or 612, userinterface 600 may be modified (either by the portable electronic deviceand/or in response to user-interface information received from the A/Vhub) to include information specifying the location of an associatedcontent source in the environment (such as in the information displayedin one of virtual command icons 610). However, in other embodimentsvirtual command icons 610 indicates available content sources to theuser without indicating the locations of the content sources. Thus, thedisplay technique may allow the portable electronic device and the A/Vhub to operate in a content-source-location independent manner, i.e.,the associated HDMI content may be displayed independently of a locationof the associated content source.

Similarly, as described further below with reference to FIGS. 18-20, theuser may use virtual command icon 614 in user interface 600 to performone-operation casting of content (such as HDMI content) from one A/Vdisplay device in the environment to another A/V display device in theenvironment. In particular, the user may be able to cast the HDMIcontent by activating virtual command icon 614. For example, if the userselects a content source that is currently providing a data stream to anA/V display device somewhere in system 100 (FIG. 1) by touching thesurface of the touch-sensitive display within a strike area associatedwith virtual command icons 610, user interface 600 may be modified(either by the portable electronic device and/or in response touser-interface information received from the A/V hub) to include virtualcommand icon 614. If the data stream with content from the selectedcontent source is currently being displayed on an A/V display device atanother location (such as in a different room), then, if the useractivates virtual command icon 614, another data stream with thiscontent may be cast and displayed on another A/V display device, such asan A/V display device that the user is controlling using the portableelectronic device. This ‘one-step casting’ (based on the activation of asingle command feature, such as a single virtual command icon orphysical button) may involve the A/V hub providing one or more commandsor instructions in response to receiving information specifying anactivation command associated with the activation of virtual commandicon 614. In some embodiments, the one or more commands includeselectively turning on the other A/V display device (if the other A/Vdisplay device is not already activated), instructing the content sourceto provide the other data stream with the selected content to the A/Vhub, etc. Thus, the one-step casting may be performed based ondevice-state information, such as device-state information about theother A/V display device.

Alternatively, the user selects a content source that is currentlyproviding a data stream to an A/V display device at the user's currentlocation, then user interface 600 may be modified (either by theportable electronic device and/or in response to user-interfaceinformation received from the A/V hub) to include information specifyingavailable A/V display devices or information associated with theavailable A/V display devices (such as locations in the environment ofthe A/V display devices). Next, if the user activates one of the virtualcommand icons associated with one of the available A/V display devices(such as a virtual command icon associated with a location, e.g., atelevision in the living room or a bedroom), the resultinguser-interface activity information from the portable electronic devicemay instruct the A/V hub to provide display instructions to thespecified A/V display device and to route or cast the HDMI content fromone of the content sources to the specified A/V display device. Onceagain, this may involve one or more commands or instructions from theA/V hub, and the one or more commands or instructions may be selectedintelligently based on device-state information. In this way, the usermay simply and intuitively cast content, such as HDMI content, from oneA/V display device to another A/V display device.

We now describe embodiments of the information displayed on the A/Vdisplay device during methods 200 (FIG. 2) and/or 400 (FIG. 4). FIG. 7presents a drawing illustrating a display 700 (which is sometimesreferred to as a ‘command window’) in an A/V display device (such as A/Vdisplay device 114-1 in FIG. 1). In particular, display 700 shows a listof available content sources 710. However, currently no HDMI content isdisplayed on display 700.

When the user activates one of the virtual command icons associated withone of content sources 710 (such as DVR), the A/V hub may providedisplay instructions and, after requesting and receiving HDMI contentfrom the one of content sources 710, may provide the HDMI content to A/Vdisplay device. As shown in FIG. 8, which presents a drawingillustrating a display 800 in the A/V display device, the HDMI contentmay be presented in a tiled window 810. Note that the list of availablecontent sources highlights the DVR under ‘current and that the remainingcontent sources are shown below the horizontal line in alphabeticalorder. Thus, the user may use the user interface on the portableelectronic device to dynamic select and set up a tiling window using onestep or operation. (However, in other embodiments, more than one step oroperation may be used.) Note that if the virtual command icon for theDVR is still selected, the user may be presented with an ‘X’ virtualcommand icon that the user can use to discontinue the display of theHDMI content in tiled window 810. Thus, one or more of virtual commandicons 612 (FIG. 6) may be dynamically displayed on display 810 based onthe user's selections and/or activation of virtual command iconsconcurrently displayed on the user interface in the portable electronicdevice.

Similarly, when the user selects another of the virtual command iconsassociated with a second one of the content sources (such as a Blue-Rayplayer) and then activates a ‘+’ virtual command icon that isdynamically displayed next to the selected other virtual command icon,the A/V hub may provide display instructions and, after requesting andreceiving additional HDMI content from the second content source, mayprovide the additional HDMI content to A/V display device. As shown inFIG. 9, which presents a drawing illustrating a display 900 in the A/Vdisplay device, the additional HDMI content may be presented in a tiledwindow 910. Note that the additional HDMI content may be presented in atiled window 910 may be presented on display 900 concurrently orsimultaneously with the HDMI content presented in tiled window 810.Thus, there may be multiple independent real-time or live video streamspresented or displayed in display 900. The user may user the userinterface on the portable electronic device to select the audio for oneof tiled windows 810 and 910, which is then output on the portableelectronic device (such as to headphones), while the audio from theother of tiled windows 810 and 910 may be output on speakers in the sameroom or another room in the environment that includes the A/V hub.

While FIG. 9 illustrates tiled windows 810 and 910 arranged along avertical direction 912, in other embodiments different spatialarrangements or configurations of tiled windows 810 and 910 are used.This is illustrated in FIG. 10, which presents a drawing illustrating adisplay 1000 in the A/V display device in which tiled windows 810 and910 are arranged along a horizontal direction 1010.

Furthermore, as discussed previously, the user may use the userinterface on the portable electronic device to discontinue the displayof HDMI content from one of the content sources. For example, aftertiled window 810 is displayed, if the user holders their finger incontact with the surface of the touch-sensitive display on the portableelectronic device within the strike area of a virtual command iconassociated with the DVR, the user interface on the portable electronicdevice and the information to be displayed on the display in the A/Vdisplay device may be modified to indicate that the user has the optionto discontinue the display of HDMI content from the one of the contentsources (such as the DVR). In particular, a virtual command iconrepresented by an ‘X’ in a circle may be displayed adjacent to thevirtual command icon for the DVR. If the user activates this virtualcommand icon, the A/V hub may provide display instructions to the A/Vdisplay device to discontinue tiled widow 810 and may instruct the DVRto discontinue providing the HDMI content. As shown in FIG. 11, whichpresents a drawing illustrating a display 1100 in the A/V displaydevice, tiled window 810 may be discontinued, so that only tiled window910 with the additional HDMI content is displayed.

In some embodiments, the user may use the user interface on the portableelectronic device to move the HDMI content displayed in a tiled windowto a central tiled window in the display in the A/V display device. Thisis illustrated in FIG. 12, which presents a drawing illustrating adisplay 1200 in the A/V display device. As shown in FIG. 12, initiallythere may be one or more tiled windows 1210 that display HDMI contentassociated with one or more of the content sources that are currentlyproviding data streams with HDMI content, such as the DVR and thestreaming TV/media player. Initially, tiled window 1210-1 may displaythe HDMI content from the DVR, and tiled window 1210-2 may display theHDMI content from the streaming TV/media player. Note that when the usercontacts the surface of the touch-sensitive display within a strike areaof the virtual command icon for the DVR and/or activates this virtualcommand icon, an additional virtual command icon (as represented by thecircle with the ‘X’) that allows the user to discontinue the displayingof tiled window 1210-1 is dynamically displayed.

Then, if the user slides over and activates the virtual command icon forthe streaming TV/media player (by contacting and releasing the surfaceof the touch-sensitive display within an associated strike area) thepositions of tiled windows 1210 may be reversed. (Note that anadditional virtual command icon that allows the user to discontinue thedisplaying of tiled window 1210-2 is also dynamically displayed.) Inparticular, in response to activating the virtual command icon for thestreaming TV/media player, the A/V hub may provide display instructionsthat transitions the associated HDMI content to a central tiled windowand discontinues the initial tiled window. The result is shown in FIG.13, which presents a drawing illustrating a display 1300 with the HDMIcontent presented or displayed in central tiled window 1210-2. Thus, theuser may use the user interface on the portable electronic device todynamically change the HDMI content displayed in the central tiledwindow.

While the preceding discussion illustrated the use of a virtual commandicons in a user interface displayed on a touch-sensitive display in theportable electronic device, in other embodiments the user may activatephysical buttons. For example, the user may point the portableelectronic device at a location on the A/V display device and may pressa button. The location on the A/V display device, which may correspondto a virtual command icon, may be identified based on an infra-redsignal from the portable display device, wireless ranging and/ororientation of the portable display device, etc. This user-interfaceinformation may be used by the A/V hub to perform one or more commandsor instructions and, as needed, to modify the displayed user interfaceon the A/V display device.

In some embodiments, tiled windows with video content are displayedwithout the user activating an associated virtual command icon or aphysical button. For example, the display of a tiled window may be eventdriven, such as when a security or intrusion-detection device is trigger(such as based on sound or motion detection) or when a baby monitordetects motion or a baby crying. In these cases, a tiled window withvideo from a security camera or a baby camera may be automaticallydisplayed based on the context. This may allow the user to visuallyassess whether further action is required.

Embodiments of the display technique are further illustrated in FIG. 14,which presents a flow diagram illustrating a method 1400 for providingdisplay instructions. This method may be performed by an A/V hub, suchas A/V hub 112 (FIG. 1). During operation, the A/V hub (such as acontrol circuit or control logic, e.g., a processor executing a programmodule and/or or a circuit) generates display instructions (operation1410) specifying information to be displayed on a display in an A/Vdisplay device, where the information to be displayed includes iconsassociated with the content sources. Note that the content sources maybe located at arbitrary locations in a structure, including locationsexternal to an environment of the A/V hub and the A/V display device.For example, the environment may include one or more rooms in thestructure, and the arbitrary locations may be located in different roomsin the structure than the A/V hub and the A/V display device.

Then, the A/V hub provides, via an interface circuit in the A/V hub, thedisplay instructions (operation 1412) to the A/V display device fordisplay on the display.

Next, the A/V hub may optionally receive, via the interface circuit,user-interface activity information (operation 1414) from a portableelectronic device that specifies user selection of a content source. Forexample, the user-interface activity information may include activationof a virtual command icon displayed in the user interface on atouch-sensitive display in the portable electronic device.Alternatively, the user-interface activity information may includeactivation or changing a state or configuration of a physical button.Note that the A/V hub may optionally: generate user-interfaceinformation that specifies the user interface that includes one or morevirtual command icons, including the virtual command icon, which areassociated with the content sources; and provides the user-interfaceinformation to the portable electronic device for display on thetouch-sensitive display in the portable electronic device. However, inother embodiments the user-interface information is optionally generatedby the portable electronic device.

In response, the A/V hub may optionally perform one or more additionaloperations (operation 1416). In particular, the A/V hub may optionally:provide, via the interface circuit, a request for HDMI content to thecontent source; selectively provide, via the interface circuit, anactivation command to the content source based on device-stateinformation of the content source (such as when the content source isnot already turned on or activated); receive, via the interface circuit,the HDMI content from the content source; and provide, the HDMI contentand second display instructions to the A/V display device as frames withthe HDMI content are received from the content source, so that the HDMIcontent is displayed on the display in the A/V display device. In someembodiments, the A/V hub may optionally determine the second displayinstructions based on a format of the display. Note that providing theHDMI content and the second display instructions involves transcodingthe HDMI content based on the format of the display.

FIG. 15 presents a drawing illustrating communication among theelectronic devices in FIG. 1, which presents a drawing illustratingcommunication between portable electronic device 110, A/V hub 112, A/Vdisplay device 114-1 and one or more content sources 116. In particular,processor 310 may determine display instructions 1510 specifyinginformation to be displayed on a display in an A/V display device 114-1.Then, processor 310 may optionally provide, via interface circuit 314,display instructions 1510 to A/V display device 114-1 for display on thedisplay, where the information to be displayed includes icons associatedwith content sources 116. Note that content sources 116 may be locatedat arbitrary locations in a structure, including locations external toan environment of A/V hub 112 and A/V display device 114-1.

Moreover, processor 310 may optionally provide, via interface circuit314, user-interface information 1512 to portable electronic device 110.After receiving user-interface information 1512, portable electronicdevice 110 may optionally display an associated user interface on atouch-sensitive display (such as touch-sensitive display 124 in FIG. 1)in portable electronic device 110. However, in other embodimentsuser-interface information 1512 is optionally generated by portableelectronic device 110.

Next, portable electronic device 110 may provide user-interface activityinformation 1514 that specifies user selection of one of content sources116 (such as user activation of a single command feature associated withthe one of content sources 116) to A/V hub 112. After receivinguser-interface activity information 1514, interface circuit 314 mayprovide user-interface activity information 1514 to processor 310. Inresponse, processor 310 may instruct interface circuit 314 to providerequest 1516 for HDMI content 1518 to the one of content sources 116. Inaddition, processor 310 may optionally determine display instructions1520 based on a format of the display in A/V display device 114-1.Alternatively, display instructions 1520 may be predetermined orpredefined.

After receiving request 1516, the one of content sources 116 may provideHDMI content 1518 to A/V hub 112. Furthermore, interface circuit 314 mayprovide HDMI content 1518 and display instructions 1520 to A/V displaydevice 114-1 as frames with HDMI content 1518 are received from the oneof content sources 116, so that HDMI content 1518 is displayed on thedisplay in A/V display device 114-1. Note that display instructions 1520may specify that HDMI content 1518 is to be displayed in a tiled windowon the display while additional HDMI content from another of contentsources 116 is displayed on the display. (Alternatively, in someembodiments interface circuit 314 provides HDMI content 1518 toprocessor 310, which instructs interface circuit 314 to provide HDMIcontent 1518 and display instructions 1520 to A/V display device 114-1as frames with HDMI content 1518 are received from the one of contentsources 116.)

In this way, the display technique may make it easier and more intuitivefor a user to control the content and the layout of the contentdisplayed on the A/V display device (such as how and where the contentis displayed). In particular, by allowing the user to see contentsources in a network throughout the environment that includes the A/Vhub regardless of their locations and/or the current state of thecontent sources (which may also be displayed), the display technique mayallow the user to select content sources at different locations in theenvironment and, thus, may provide location-independent operation. Inthe process, the display technique may reduce user frustration, and thusmay improve user satisfaction when using the portable electronic device,the A/V hub, the A/V display device and/or the one or more contentsources.

We now describe embodiments of the information displayed on the A/Vdisplay device (in conjunction with one or more virtual command icons inthe user interface on the portable electronic device) during method 1400(FIG. 14). FIG. 16 presents a drawing illustrating a display 1600 in anA/V display device (such as A/V display device 114-1 in FIG. 1). Inparticular, display 1600 shows a list of available content sources 1610at different locations in the environment.

When the user activates one of the virtual command icons associated withone of content sources 1610 (such as the DVR in the bedroom), the A/Vhub may provide display instructions and, after requesting and receivingHDMI content from the one of content sources 1610, may provide the HDMIcontent to A/V display device. As shown in FIG. 17, which presents adrawing illustrating a display 1700 in the A/V display device, the HDMIcontent may be presented in a tiled window 1710.

Embodiments of the display technique are further illustrated in FIG. 18,which presents a flow diagram illustrating a method 1800 for casting ofHDMI content. This method may be performed by an A/V hub, such as A/Vhub 112 (FIG. 1). During operation, the A/V hub (such as a controlcircuit or control logic, e.g., a processor executing a program moduleand/or or a circuit) receives, via an interface circuit in the A/V hub,user-interface activity information (operation 1810) from a portableelectronic device, where the user-interface activity informationspecifies a user instruction to cast the HDMI content associated with acontent source from a first A/V display device to a second A/V displaydevice (such as from a computer to a television). For example, theuser-interface activity information may include activation of one ormore virtual command icons (such as a single virtual command icon)displayed in the user interface on a touch-sensitive display in theportable electronic device. Alternatively, the user-interface activityinformation may include activation or changing a state or configurationof one or more physical buttons (such as a single physical button). Notethat the A/V hub may optionally: generate user-interface informationthat specifies the user interface that includes one or more virtualcommand icons, including the single virtual command icon, which areassociated with the first A/V display device and the second A/V displaydevice; and provides the user-interface information to the portableelectronic device for display on the touch-sensitive display in theportable electronic device. However, in other embodiments theuser-interface information is optionally generated by the portableelectronic device. Note that the first A/V display device and the secondA/V display device may be associated with different providers.

In response, the A/V hub accesses device-state information (operation1812) that specifies a current state of the second A/V display device.For example, as described further below with reference to FIGS. 22 and23, the device-state information may be determined using astate-detection circuit that is coupled to at least a pin in a port inthe A/V hub. Alternatively or additionally, the device-state informationmay be inferred based on device commands previously provided to thesecond A/V display device and a device-state diagram. In someembodiments, the device-state information includes a history of one ormore most-recent device commands previously provided to and received bythe second A/V display device (and, more generally, an entertainmentdevice) and/or a current state of the entertainment device based on thehistory of the one or more most-recent device commands and adevice-state diagram of the second A/V display device (and, moregenerally, the entertainment device). In some embodiments, thedevice-state information is stored in a computer-readable medium, suchas a memory.

Then, the A/V hub performs the casting (operation 1814) of the HDMIcontent from the first A/V display device to the second A/V displaydevice using a dynamic number of operations based on the device-stateinformation, so that the HDMI content is displayed on a display in thesecond A/V display device (which may be provided by a differentmanufacturer than the first A/V display device). Note that, from theuser perspective, the casting operation is from the first A/V displaydevice to the second A/V display device. However, in practice, thecasting operation involves the content source providing the HDMI contentto the second A/V display device. After the casting operation, the HDMIcontent may or may not still be displayed on the first A/V displaydevice.

In general, casting may include operations such as: turn on or power onthe A/V hub; turn on or power on the second A/V display device; set tocorrect input or content source; and/or enable casting. Once the userinstruction is received, the casting may occur in an automated manner.Note that when the device-state information indicates that the secondA/V display device is already powered on, a power-on operation may beexcluded from the dynamic number of operations. Moreover, when thedevice-state information indicates that the second A/V display device isalready set to receive the HDMI content from the content source, anoperation to set the second A/V display device to receive the HDMIcontent from the content source may be excluded from the dynamic numberof operations. In some embodiments, the casting may include receivingthe HDMI content from the content source, transcoding the HDMI contentbased on a format of a display in the second A/V display device, andproviding the transcoded HDMI content to the second A/V display device.

Note that the casting may be performed without further user action afteractivating the single virtual command icon or the single physicalbutton. (Thus, method 1800 may be used by the user to perform so-calledone-step or one-operation casting, even though it may involve the A/Vhub performing multiple commands or instructions.)

FIG. 19 presents a drawing illustrating communication among theelectronic devices in FIG. 1, which presents a drawing illustratingcommunication between portable electronic device 110, A/V hub 112, A/Vdisplay devices 114-1 and A/V display devices 114-2. In particular,processor 310 may optionally provide, via interface circuit 314,user-interface information 1910 to portable electronic device 110. Afterreceiving user-interface information 1910, portable electronic device110 may optionally display an associated user interface on atouch-sensitive display (such as touch-sensitive display 124 in FIG. 1)in portable electronic device 110. However, in other embodimentsuser-interface information 1910 is optionally generated by portableelectronic device 110.

Next, portable electronic device 110 may provide to A/V hub 112user-interface activity information 1912 (such as user activation of asingle virtual command icon) that specifies a user instruction to castthe HDMI content associated with a content source from A/V displaydevice 114-1 to A/V display device 114-2. After receiving user-interfaceactivity information 1912, interface circuit 314 may provideuser-interface activity information 1912 to processor 310.

In response, processor 310 may access device-state information 1914 ofA/V display device 114-2 in memory 1916 and/or may measure or determinedevice-state information 1914 using state-detection circuit 1918, adevice-state diagram and/or a history of previous device commandsprovided to and/or received by A/V display device 114-2. Based ondevice-state information 1914, processor 310 may dynamically adapt orchange a number of operations in a casting process 1920 based ondevice-state information 1914. Next, processor 310 may provide, viainterface circuit 314, display instructions 1922 to A/V display devices114-1 and display instructions 1924 to A/V display devices 114-2 toperform casting process 1920. Note that processor 310 may optionallydetermine display instructions 1924 based on a format of the display inA/V display devices 114-2. Alternatively, display instructions 1924 maybe predetermined or predefined.

In this way, the display technique may make it easier and more intuitivefor a user to control the content and the layout of the contentdisplayed on the A/V display device (such as how and where the contentis displayed). In particular, the display technique may allow the user,with minimum effort or operations, to cast content (such as HDMIcontent) from one A/V display device to another A/V display device. Inthe process, the display technique may reduce user frustration, and thusmay improve user satisfaction when using the portable electronic device,the A/V hub, and/or one or more A/V display devices.

Consequently, methods 200 (FIG. 2), 400 (FIG. 4), 1400 (FIG. 14) and/or1800 (FIG. 18) may reduce user errors or mistakes when using the userinterface, which may improve the user experience when using the portableelectronic device and/or the A/V hub.

In some embodiments of methods 200 (FIG. 2), 400 (FIG. 4), 1400 (FIG.14) and/or 1800 (FIG. 18), there may be additional or fewer operations.Moreover, the order of the operations may be changed, and/or two or moreoperations may be combined into a single operation. Furthermore, one ormore operations may be modified. For example, instead of involving theactivation of a single feature (such as a single virtual command icons),in some embodiments one or more of the operations involve the activationof one or more features (such as one or more virtual command icons).Alternatively or additionally, display instructions may be provided toan A/V display device differentially (such as when the displayinstructions change), regularly or periodically (such as in one of everyN packets or in a packet in each frame) or in each packet.

Note that in this display technique the A/V hub may display the HDMIcontent (and, more generally, video content) to an arbitrary A/V displaydevice (including an A/V display device that is located remotely fromthe A/V hub, such as in another room) without a need for a separateset-top box that is located proximate to the A/V display device.Instead, the A/V hub may perform all of the frame-by-frame transcodingof the video content that is needed for the A/V display device todisplay the video content before providing the video content to the A/Vdisplay device. Thus, in contrast with many existing cable and satellitesystems, the A/V hub may provide video content to multiple A/V displaydevices (such as N A/V display devices) without the use of N associatedset-top boxes. Consequently, the A/V hub may eliminate the need for aseparate set-top box in the same room as an A/V display device (althoughthere may be a local wireless receiver that is associated with the A/Vhub). This capability may be enabled by the knowledge of the devicestate information and the content selected by the users that isavailable to the A/V hub. In addition, this capability may eliminate theneed for a user to know where or how a particular A/V display device isconnected to a content source, such as cable television or a satellitedish.

We now describe embodiments of the information displayed on the A/Vdisplay device (in conjunction with one or more virtual command icons inthe user interface on the portable electronic device) during method 1800(FIG. 18). FIG. 20 presents a drawing illustrating a display 2000 in anA/V display device (such as A/V display device 114-1 in FIG. 1). Inparticular, after selecting a virtual command icon associated with anactive content source that is currently streaming content to the A/Vdisplay device and activating an associated casting virtual commandicon, display 2000 may show a list of A/V display devices 2010 at otherlocations. If the user selects another A/V display device in the list ofA/V display devices 2010 (such as A/V display device 114-2 in FIG. 1) byactivating a single virtual command icon in the user interface on theportable electronic device, at least some of the HDMI content displayedon the A/V display device (such as the HDMI content in optional tiledwindow 2012) may be cast to other A/V display device in the list of A/Vdisplay devices 2010. This casting may involve the A/V hub providing oneor more commands or instructions, including selectively turning on theother A/V display device (if it is not already turned on or activated),instructing the content source to provide a data stream with the HDMIcontent to the other A/V display device, etc. Thus, the one or morecommands or instructions may be based on device-state information aboutthe other A/V display device and the content source. Note that the HDMIcontent may or may not still be displayed in optional tiled window 2012.For example, when the casting operation is specified optional tiledwindow 2012 may no longer be displayed. Alternatively, optional tiledwindow 2012 may be displayed until the user activates a virtual commandicon to discontinue the displaying of the HDMI content.

We now describe embodiments of generating an output data stream from A/Vhub 112 in FIG. 1. FIG. 21 presents a drawing illustrating A/V hub 112.In A/V hub 112, one or more input data streams are received from one ormore content sources. In particular, the one or more input data streamswith video content that is compatible with or corresponds to (such as ifat least one of the input data streams is encoded and/or compressed) ahigh-definition format (such as 1080p and/or HDMI) may be received byinterface circuit 2110 via one or more input ports 2112 (such asHDMI-compatible input ports) and/or one or more antennas 2114. Forexample, the one or more input data streams may include full frames ofvideo content from the one or more content sources.

Moreover, one of the input data streams may be input directly to anintegrated circuit (I.C.) 2116 (such as an integrated circuit fromBroadcom). Additional input data streams in the one or more inputstreams may be input to integrated circuits 2118 (such as via a controllogic 2124, e.g., an FPGA, an integrated circuit from Cavium, etc.), andthe output transport streams from integrated circuits 2118 may be inputto integrated circuit 2116 (e.g., using a real-time transport protocol).(Note that connections to memory 2122 and control logic 2124 are notshown in FIG. 21.) As described further below, integrated circuit 2116may generate an output data stream, which may be provided, via interfacecircuit 2126, to output port 2128 (such as an HDMI-compatible outputport) and/or one or more antennas 2130 for an A/V display device, suchas A/V display device 114-1 (FIG. 1). This output data stream mayinclude information associated with frames of scaled video content forthe one or more input data streams, i.e., the output data stream maytemporally multiplex the spatial information for different scaled videocontent, as well as display instructions that specify how the frames ofscaled video content are to be displayed on the A/V display device. Insome embodiments, optional digital-to-analog (A/D) integrated circuit2120 may mirror the output data stream into two identical output datastreams.

Note that integrated circuits 2118 (or other integrated circuits notshown) may decode the additional input data streams into separatebuffers in memory 2122. Moreover, in embodiments where an input datastream is encrypted (such as using high-bandwidth digital contentprotection), it may be decrypted by one of integrated circuits 2118 (orby another integrated circuit not shown). Furthermore, integratedcircuits 2118 may re-encode the video content using MPEG-4 Part 4encoding (which is sometimes referred to as ‘H.264’). However, audiocontent may be simply re-multiplexed into the transport stream and sentto integrated circuit 2116.

While integrated circuit 2116 may directly receive one input data streamthat has the HDMI format or a YUV format, the additional input datastreams may be MPEG-4 Part 4 encoded. Consequently, integrated circuit2116 may first decode the additional input data streams. Then,integrated circuit 2116 may optionally recode three composite channels(such as RGB or YUV) for each of the video content into a singleplane/buffer (such as a graphics plane) and, thus, into the output datastream. For local viewing, integrated circuit 2116 may output the outputdata stream to output port 2128. Thus, in some embodiments the outputdata stream may have the HDMI format (and may not be re-encoded orcompressed). However, for remote viewing, the one or more antennas 2114may be used. In these embodiments, the frames of scaled video content inthe output data stream may be encoded using MPEG-4 Part 4 encoding.

In some embodiments, the re-encoding by integrated circuit 2116 andintegrated circuits 2118 is variable bit rate with a maximum bit rate of25 MBps. The frames of video content may also be an open group ofpictures. For example, there may be one I frame (which is also the IDRframe), followed by several P frames. This pattern may be repeatedmultiple times as needed, or there may be additional P frames as needed.Note that when an electronic device (such as the A/V display devicerequests a new IDR frame (such as when some packets have been dropped),a new IDR frame may be encoded, and then once again followed by Pframes.

Note that the temporally multiplexed frames of scaled video content inthe output data stream may have a data rate that is less than athreshold value associated with swapping data in memory 2122. Forexample, memory 2122 may include DDR3 and the threshold value may be 25Mb/s. DDR memory usually limits how much data in memory it can help moveacross the system. 25 Mb/s is a typical encoding bitrate without loss ofimage quality. In particular, 25 Mb/s may offer a reasonable compressionof video without much loss in fidelity. Stated differently, DDR memoryusually has a bandwidth constraint, which is why the data in A/V hub 112is compressed. In this system, there are two kinds of memory: the systemmemory (which is used by the rest of the operating system to route data,handle network traffic, for user-interface information, etc.), and thevideo memory (which is owned and managed by a ‘fast channel’ or ‘videonetwork’). In particular, a DDR video network may be used to moveuncompressed video data from the decoder output to the graphicscompositor/display or the graphics plane. The fast past may start at theoutput of decoders (or HDMI uncompressed input), and may end at thegraphics compositor. The video network may rely on a dedicated memorybank for specifically moving uncompressed audio/video data. This mayensure that data is moved using direct memory access across memory withan expected performance guarantee. However, the video network may beless flexible than the system memory in terms of what can be done withit. Therefore, it may be used to store the decoded frame, so the scalercan use it, then the scaled frame so the compositor can use it. Incontrast, the system memory can be used to move memory from any sourceto any destination (e.g., from an input source to a network socket, orfrom a socket to a file, etc.) This is why it may be easier to deal withcompressed/encoded video data in the system-memory domain, anduncompressed data in the video-network (video-memory) domain.

For an input data stream with a full frame of video content having ahigh-definition format (such as 1080p), there may be 1920×1080bits/channel. Therefore, with three channels, there may be 6.22Mb/frame. With 60 frames/s, the date rate for one unencoded oruncompressed input data stream with this video content may be 373.248Mb/s. Consequently, in order to obtain the data rate of 25 Mb/s, thisvideo content may be scaled and/or compressed by some 15× in the outputdata stream, so that each frame is some 416.6 k. With additional inputdata streams, data rate for the scaled video content in the output datastream may be adjusted or selected accordingly. For example, with thefirst tiled window have a size that is 2× that of the second tiledwindow, the resulting data rates for the scaled video content may be,respectively, less than 16.67 Mb/s and 8.33 Mb/s.

Moreover, the video content for the different input data streams may bescaled by integrated circuit 2116 based on the data-rate constraint andthe display instructions. For example, the display instructions mayspecify two tiled windows that are associated, respectively, with firstvideo content and second video content in two input data streams. Thetiled windows may have a common size, or one may have a size that is afraction of the size of the other tiled window (such as 0.5× or 0.25×the size of the other tiled window). Integrated circuit 2116 may scalethe first video content and second video content (to generate firstscaled video content and second scaled video content) based on the sizesof the tiled windows and the threshold value of the data rate of theoutput data stream.

Similarly, if a user uses portable electronic device 110 (such as byactivating a virtual command icon in a user interface) to select anothercontent source, A/V hub 112 may provide, via interface circuit 2116, arequest to the other content source for a third vide stream, and thenmay receive, via interface circuit 2110, a third input data stream thatincludes frames of the third video content, which may correspond to thehigh-definition format. In this case, with three input data streams, thedisplay instructions may specify three associated tiled windows. Thetiled windows may have a common size, or the sizes of two of the tiledwindows may be a fraction of the size of the other tiled window (such as0.25× the size of the other tiled window). Integrated circuit 2116 mayscale the first video content, the second video content and the thirdvideo content (to generate first scaled video content, second scaledvideo content and third scaled video content) based on the sizes of thetiled windows and the threshold value of the data rate of the outputdata stream.

While memory 2122 is shown as a separate, discrete component in FIG. 21(which may be accessed by integrated circuit 2116, integrated circuits2118 and/or control logic 2124), in other embodiments memory 2122 isdistributed in A/V hub 112 and/or may be included in or proximate toother components, such as integrated circuit 2116, integrated circuits2118 and/or control logic 2124.

As noted previously, the device-state information (such as whether anentertainment device is: electrically coupled to A/V hub 112 in FIG. 1,in a power-on state, in a power-off state, and/or another state, e.g., aplayback state, a pause state, a stop state, etc.) may be determinedusing hardware (such as a state-detection circuit) and/or software(which may be executed by a processor and, more generally, a controlcircuit). FIG. 22 presents a block diagram illustrating astate-detection circuit 2210 in A/V hub 112 (FIG. 1). In A/V hub 112(FIG. 1), input connector 2212 (which may be compatible with an HDMIstandard) may be electrically coupled to an entertainment device.State-detection circuit 2210 may be coupled to at least pin 2214 ininput connector 2212, so that, when the entertainment device iselectrically coupled to input connector 2212, state-detection circuit2210 establishes a ground loop between A/V hub 112 (FIG. 1) and theentertainment device. For example, pin 2214 may include a transitionminimized differential signaling (TMDS) data1 shield. (Alternatively,pin 2214 may include a TMDS data1 shield.) Moreover, state-detectioncircuit 2210 may include: an energy-dissipation component (such asresistor 2216) electrically coupled to a power-supply voltage and pin2214 (which may provide electrostatic-discharge protection); anenergy-storage component (such as capacitor 2218) electrically coupledto pin 2214 and ground; and a bi-directional voltage clamp (such asvaristor 2220 or a Verner diode), in parallel with capacitor 2218,electrically coupled to pin 2214 and ground. For example, resistor 2216may be 150 kΩ and capacitor 2218 may be 0.047 μF. In some embodiments,state-detection circuit 2210 includes a general-purpose input/output(GPIO) device 2226 coupled to pin 2214. The behavior (such as an inputpin, an output pin, enabled or disabled) of GPIO device 2226 may becontrolled using control signals or instructions from control logic2224.

FIG. 23 presents a flow diagram illustrating a method 2300 for detectingan entertainment device, which may be performed by an A/V hub (such asA/V hub 112 in FIG. 1) using state-detection circuit 2210 in FIG. 22.During operation, a control circuit (such as a processor and/or controllogic 2224, which may be included in or external to state-detectioncircuit 2210) in A/V hub 112 (FIG. 1), which is electrically coupled toinput connector 2212, detects whether there is electrical couplingbetween the entertainment device and input connector 2212 usingstate-detection circuit 2210 (FIG. 22). In particular, detecting whetherthere is electrical coupling between the entertainment device and inputconnector 2212 may involve: setting pin 2214 as an input (operation2310), where pin 2214 is then pulled to a power-supply voltage bycontrol logic 2224; measuring a voltage on pin 2214 (operation 2312)using control logic 2224; and detecting the electrical coupling betweenthe entertainment device and input connector 2212 when the voltage onpin 2214 is less than or equal to a predefined value (operation 2314),such as when the voltage is approximately ground, using control logic2224. Note that, when the electrical coupling between the entertainmentdevice and input connector 2212 is detected, control logic 2224 may: setpin 2214 as an output and electrically couple pin 2214 to ground(operation 2318), which may improve signal integrity; and measure asecond voltage (operation 2320) on hotplug-detect pin 2222 in inputconnector 2212. When the second voltage on hotplug-detect pin 2222 isless than or equal to the predefined value (operation 2322), controllogic 2224 may set pin 2214 as an input (operation 2324) and repeat themeasurement of the voltage on pin 2214 (operation 2312). Alternatively,when the voltage equals or exceeds a second predefined value (operation2326), such as when the voltage is approximately the power-supplyvoltage, control logic 2224 may repeat detecting whether there iselectrical coupling between the entertainment device and input connector2212. Furthermore, when the voltage is less than or equal to thepredefined value (operation 2314), control logic 2224 may identify acurrent state (operation 2316) of the entertainment device, such as: apower-off state, and a standby state. For example, control logic 2224may provide the set of first control commands, provide the set of secondcontrol commands, and/or may monitor (via one or more pins in inputconnector 2212) content activity, such as a data stream to and/or fromthe entertainment device. Thus, control logic 2224 may determine thatthe entertainment device is: in the power-off state when there is no adata stream; in the standby state when the data stream has a low datarate; and in the playback state when the data stream has a data rateassociated with A/V content and/or includes the A/V content. Note that,when the second voltage on hotplug-detect pin 2222 is less than or equalto the predefined value (operation 2322) and when the voltage is lessthan or equal to the predefined value (operation 2314), control logic2224 may repeat setting pin 2214 as the output and electrically couplingpin 2214 to ground (operation 2318).

When the electrical coupling between the entertainment device and inputconnector 2212 is detected, control logic 2224 may optionally attempt toidentify the entertainment device by providingconsumer-electronics-control commands (which may be compatible with anHDMI standard) to the entertainment device. Alternatively oradditionally (such as when the attempt is unsuccessful), control logic2224 may provide a set of first control commands associated withdifferent types of entertainment devices until, in response, contentactivity (such as packets or frames associated with a data stream ofcontent communicated to and/or from the entertainment device) isdetected by control logic 2224 via input connector 2212. For example,the set of first commands may include: a play command for the differenttypes of entertainment devices; and/or a trick-mode command (such asfast forward, reverse, fast reverse, or skip) for the different types ofentertainment devices. Moreover, when the content activity is detected,control logic 2224 may provide a set of second control commandsassociated with different providers of entertainment devices until achange in a state of the entertainment device is detected by controllogic 2224 via input connector 2212 and state-detection circuit 2210.The set of second control commands may include: power-on controlcommands for the different providers of entertainment devices; and/orpower-off control commands for the different providers of entertainmentdevices.

Alternatively or additionally, during operation control logic 2224 maydetect whether there is electrical coupling between the entertainmentdevice and input connector 2212 using state-detection circuit 2210 (FIG.22). When the electrical coupling between the entertainment device andinput connector 2212 is detected, control logic 2224 may: set pin 2214as an output and electrically couple pin 2214 to ground; and measure thesecond voltage on hotplug-detect pin 2222 in input connector 2212. Whenthe second voltage on hotplug-detect pin 2222 is less than or equal tothe predefined value, control logic 2224 may set pin 2214 as an inputand measure a voltage on pin 2214. Moreover, when the voltage equals orexceeds the second predefined value, control logic 2224 may repeatdetecting whether there is electrical coupling between the entertainmentdevice and input connector 2212. Furthermore, when the voltage is lessthan or equal to the predefined value, control logic 2224 may identifythe current state of the entertainment device. In some embodiments,control logic 2224: provides a control command to the entertainmentdevice; and identifies an additional state of the entertainment devicebased on content (such as A/V content) that is provided and/or receivedby the entertainment device in response to the control command. Forexample, the control command may include: a play command, and/or atrick-mode command (such as fast forward or fast skip, slow forward orslow skip, fast reverse, or slow reverse).

We now describe embodiments of an electronic device. FIG. 24 presents ablock diagram illustrating an electronic device 2400, such as portableelectronic device 110, A/V hub 112 or A/V display device 114-1 inFIG. 1. This electronic device includes processing subsystem 2410,memory subsystem 2412, networking subsystem 2414 and optional feedbacksubsystem 2434. Processing subsystem 2410 includes one or more devicesconfigured to perform computational operations. For example, processingsubsystem 2410 can include one or more microprocessors,application-specific integrated circuits (ASICs), microcontrollers,programmable-logic devices, and/or one or more digital signal processors(DSPs). One or more of these components in processing subsystem aresometimes referred to as a ‘control circuit.’

Memory subsystem 2412 includes one or more devices for storing dataand/or instructions for processing subsystem 2410 and networkingsubsystem 2414. For example, memory subsystem 2412 can include dynamicrandom access memory (DRAM), static random access memory (SRAM), and/orother types of memory. In some embodiments, instructions for processingsubsystem 2410 in memory subsystem 2412 include: one or more programmodules or sets of instructions (such as program module 2422 oroperating system 2424), which may be executed by processing subsystem2410. Note that the one or more program modules may constitute acomputer-program mechanism, such as a computer program or software.Moreover, instructions in the various modules in memory subsystem 2412may be implemented in: a high-level procedural language, anobject-oriented programming language, and/or in an assembly or machinelanguage. Furthermore, the programming language may be compiled orinterpreted, e.g., configurable or configured (which may be usedinterchangeably in this discussion), to be executed by processingsubsystem 2410.

In addition, memory subsystem 2412 can include mechanisms forcontrolling access to the memory. In some embodiments, memory subsystem2412 includes a memory hierarchy that comprises one or more cachescoupled to a memory in electronic device 2400. In some of theseembodiments, one or more of the caches is located in processingsubsystem 2410.

In some embodiments, memory subsystem 2412 is coupled to one or morehigh-capacity mass-storage devices (not shown). For example, memorysubsystem 2412 can be coupled to a magnetic or optical drive, asolid-state drive, or another type of mass-storage device. In theseembodiments, memory subsystem 2412 can be used by electronic device 2400as fast-access storage for often-used data, while the mass-storagedevice is used to store less frequently used data.

Networking subsystem 2414 includes one or more devices configured tocouple to and communicate on a wired and/or wireless network (i.e., toperform network operations), including: control logic 2416, interfacecircuits 2418 and associated antennas 2420. (While FIG. 24 includesantennas 2420, in some embodiments electronic device 2400 includes oneor more nodes, such as nodes 2408, e.g., pads, which can be coupled toantennas 2420. Thus, electronic device 2400 may or may not includeantennas 2420.) For example, networking subsystem 2414 can include aBluetooth networking system, a cellular networking system (e.g., a 3G/4Gnetwork such as UMTS, LTE, etc.), a universal serial bus (USB)networking system, a networking system based on the standards describedin IEEE 802.11 (e.g., a Wi-Fi networking system), an Ethernet networkingsystem, and/or another networking system. Note that the combination of agiven one of interface circuits 2418 and at least one of antennas 2420may constitute a radio. In some embodiments, networking subsystem 2414includes a wired interface, such as HDMI interface 2430 (which mayinclude a state-detection circuit).

Networking subsystem 2414 includes processors, controllers,radios/antennas, sockets/plugs, and/or other devices used for couplingto, communicating on, and handling data and events for each supportednetworking system. Note that mechanisms used for coupling to,communicating on, and handling data and events on the network for eachnetwork system are sometimes collectively referred to as a ‘networkinterface’ for the network system. Moreover, in some embodiments a‘network’ between the electronic devices does not yet exist. Therefore,electronic device 2400 may use the mechanisms in networking subsystem2414 for performing simple wireless communication between the electronicdevices, e.g., transmitting advertising or beacon frames and/or scanningfor advertising frames transmitted by other electronic devices asdescribed previously.

Within electronic device 2400, processing subsystem 2410, memorysubsystem 2412, networking subsystem 2414 and optional feedbacksubsystem 2434 are coupled together using bus 2428. Bus 2428 may includean electrical, optical, and/or electro-optical connection that thesubsystems can use to communicate commands and data among one another.Although only one bus 2428 is shown for clarity, different embodimentscan include a different number or configuration of electrical, optical,and/or electro-optical connections among the subsystems.

In some embodiments, electronic device 2400 includes a display subsystem2426 for displaying information on a display (such as the communicationwarning message), which may include a display driver, an I/O controllerand the display. Note that a wide variety of display types may be usedin display subsystem 2426, including: a two-dimensional display, athree-dimensional display (such as a holographic display or a volumetricdisplay), a head-mounted display, a retinal-image projector, a heads-updisplay, a cathode ray tube, a liquid-crystal display, a projectiondisplay, an electroluminescent display, a display based on electronicpaper, a thin-film transistor display, a high-performance addressingdisplay, an organic light-emitting diode display, a surface-conductionelectronic-emitter display, a laser display, a carbon-nanotube display,a quantum-dot display, an interferometric modulator display, amulti-touch touchscreen (which is sometimes referred to as atouch-sensitive display), and/or a display based on another type ofdisplay technology or physical phenomenon.

Furthermore, optional feedback subsystem 2434 may include one or moresensor-feedback mechanisms or devices, such as: a vibration mechanism ora vibration actuator (e.g., an eccentric-rotating-mass actuator or alinear-resonant actuator), a light, one or more speakers, etc., whichcan be used to provide feedback to a user of electronic device 2400(such as sensory feedback about the status of a user instruction tochange the state of one of the components in system 100 in FIG. 1).

Electronic device 2400 can be (or can be included in) any electronicdevice with at least one network interface. For example, electronicdevice 2400 can be (or can be included in): a desktop computer, a laptopcomputer, a subnotebook/netbook, a server, a tablet computer, asmartphone, a cellular telephone, a consumer-electronic device (such asa television, a set-top box, audio equipment, video equipment, etc.), aremote control, a portable computing device, an access point, a router,a switch, communication equipment, test equipment, and/or anotherelectronic device.

Although specific components are used to describe electronic device2400, in alternative embodiments, different components and/or subsystemsmay be present in electronic device 2400. For example, electronic device2400 may include one or more additional processing subsystems, memorysubsystems, networking subsystems, and/or display subsystems. Moreover,while one of antennas 2420 is shown coupled to a given one of interfacecircuits 2418, there may be multiple antennas coupled to the given oneof interface circuits 2418. For example, an instance of a 3×3 radio mayinclude three antennas. Additionally, one or more of the subsystems maynot be present in electronic device 2400. Furthermore, in someembodiments, electronic device 2400 may include one or more additionalsubsystems that are not shown in FIG. 24. Also, although separatesubsystems are shown in FIG. 24, in some embodiments, some or all of agiven subsystem or component can be integrated into one or more of theother subsystems or component(s) in electronic device 2400. For example,in some embodiments program module 2422 is included in operating system2424.

Moreover, the circuits and components in electronic device 2400 may beimplemented using any combination of analog and/or digital circuitry,including: bipolar, PMOS and/or NMOS gates or transistors. Furthermore,signals in these embodiments may include digital signals that haveapproximately discrete values and/or analog signals that have continuousvalues. Additionally, components and circuits may be single-ended ordifferential, and power supplies may be unipolar or bipolar.

An integrated circuit may implement some or all of the functionality ofnetworking subsystem 2414, such as one or more radios. Moreover, theintegrated circuit may include hardware and/or software mechanisms thatare used for transmitting wireless signals from electronic device 2400and receiving signals at electronic device 2400 from other electronicdevices. Aside from the mechanisms herein described, radios aregenerally known in the art and hence are not described in detail. Ingeneral, networking subsystem 2414 and/or the integrated circuit caninclude any number of radios.

In some embodiments, networking subsystem 2414 and/or the integratedcircuit include a configuration mechanism (such as one or more hardwareand/or software mechanisms) that configures the radios to transmitand/or receive on a given channel (e.g., a given carrier frequency). Forexample, in some embodiments, the configuration mechanism can be used toswitch the radio from monitoring and/or transmitting on a given channelto monitoring and/or transmitting on a different channel. (Note that‘monitoring’ as used herein comprises receiving signals from otherelectronic devices and possibly performing one or more processingoperations on the received signals, e.g., determining if the receivedsignal comprises an advertising frame, calculating a performance metric,performing spectral analysis, etc.) Furthermore, networking subsystem2414 may include at least one port (such as an HDMI port 2432) toreceive and/or provide the information in the data stream to A/V displaydevice 114-1 (FIG. 1) and/or one of the one or more content sources 116(FIG. 1).

While a communication protocol compatible with Wi-Fi was used as anillustrative example, the described embodiments may be used in a varietyof network interfaces. Furthermore, while some of the operations in thepreceding embodiments were implemented in hardware or software, ingeneral the operations in the preceding embodiments can be implementedin a wide variety of configurations and architectures. Therefore, someor all of the operations in the preceding embodiments may be performedin hardware, in software or both. For example, at least some of theoperations in the display technique may be implemented using programmodule 2422, operating system 2424 (such as drivers for interfacecircuits 2418) and/or in firmware in interface circuits 2418.Alternatively or additionally, at least some of the operations in thedisplay technique may be implemented in a physical layer, such ashardware in interface circuits 2418.

Moreover, while the preceding embodiments included a touch-sensitivedisplay in the portable electronic device that the user touches (e.g.,with a finger or digit, or a stylus), in other embodiments the userinterface is display on a display in the portable electronic device andthe user interacts with the user interface without making contact ortouching the surface of the display. For example, the user's interact(s)with the user interface may be determined using time-of-flightmeasurements, motion sensing (such as a Doppler measurement) or anothernon-contact measurement that allows the position, direction of motionand/or speed of the user's finger or digit (or a stylus) relative toposition(s) of one or more virtual command icons to be determined. Inthese embodiments, note that the user may activate a given virtualcommand icon by performing a gesture (such as ‘tapping’ their finger inthe air without making contact with the surface of the display). In someembodiments, the user navigates through the user interface and/oractivates/deactivates functions of one of the components in system 100(FIG. 1) using spoken commands or instructions (i.e., via voicerecognition) and/or based on where they are looking in the visualfeedback displayed on A/V display device 114-1 in FIG. 1 (e.g., bytracking the user's gaze or where the user is looking).

Furthermore, while A/V hub 112 (FIG. 1) was illustrated as a separatecomponent from A/V display device 114-1 (FIG. 1), in some embodimentsthe components are combined into a single component or a singleelectronic device.

While the preceding embodiments illustrated the display technique withaudio and video content (such as HDMI content), in other embodiments thedisplay technique is used in the context of an arbitrary type of data orinformation. For example, the display technique may be used withhome-automation data. In these embodiments, A/V hub 112 (FIG. 1) mayfacilitate communication among and control of a wide variety ofelectronic devices, including electronic devices in addition to or otherthan entertainment devices. Thus, A/V hub 112 (FIG. 1) and the displaytechnique may be used to facilitate or implement the so-called Internetof things.

Moreover, in the display technique different types of content may betreated similarly. For example, audio may be output and a correspondingimage or icon may be displayed in a tiled window. Furthermore, the otheroperations (such as casting) associated with the user interface and theinformation displayed on the A/V display device may have the same lookand feel and functionality as with HDMI content.

In the preceding description, we refer to ‘some embodiments.’ Note that‘some embodiments’ describes a subset of all of the possibleembodiments, but does not always specify the same subset of embodiments.

The foregoing description is intended to enable any person skilled inthe art to make and use the disclosure, and is provided in the contextof a particular application and its requirements. Moreover, theforegoing descriptions of embodiments of the present disclosure havebeen presented for purposes of illustration and description only. Theyare not intended to be exhaustive or to limit the present disclosure tothe forms disclosed. Accordingly, many modifications and variations willbe apparent to practitioners skilled in the art, and the generalprinciples defined herein may be applied to other embodiments andapplications without departing from the spirit and scope of the presentdisclosure. Additionally, the discussion of the preceding embodiments isnot intended to limit the present disclosure. Thus, the presentdisclosure is not intended to be limited to the embodiments shown, butis to be accorded the widest scope consistent with the principles andfeatures disclosed herein.

What is claimed is:
 1. An electronic device, comprising: an input port;an output port; one or more antennas; an interface circuitcommunicatively coupled to the input port, the output port and the oneor more antennas; memory configured to store uncompressed video, whereinthe memory is configured for use in dedicated video processing of theuncompressed video and has direct memory access; second memoryconfigured to store an operating system and compressed video: and one ormore integrated circuits, coupled to the interface circuit and thememory, configured to: receive at least two input data streams thatcomprise full frames of first video content and full frames of secondvideo content, wherein the input data streams correspond to ahigh-definition format; generate, using the memory, an output datastream, having a data rate that is less than a threshold valueassociated with swapping data in the memory and that is compatible witha high-definition multimedia-interface (HDMT) format, and whichcomprises temporally multiplexed frames of scaled first video contentand frames of scaled second video content, and display instructions thatspecify a first tiled window associated with the scaled first videocontent and a second tiled window associated with the scaled secondvideo content wherein the scaled first video content and the scaledsecond video content have the high-definition format and comprisecomponent channels, and wherein the generating comprises compression ofthe first video content to create the first scaled video content andcompression of the second video content to create the second scaledvideo content; and provide, via the interface circuit, the output datastream for a display device, wherein the generating comprises generatingthe scaled first video content by scaling the first video content basedon a size of the first tiled window, and generating the scaled secondvideo content by scaling the second video content based on a size of thesecond tiled window.
 2. The electronic device of claim 1, wherein one ofthe input data streams is encoded; and wherein the generating of theoutput data stream comprises decoding and decompressing the one of theinput data streams.
 3. The electronic device of claim 1, wherein the oneof the input data streams is encoded using MPEG-4 Part 4 encoding. 4.The electronic device of claim 1, wherein the memory comprises DDR3 andthe threshold value is 25 Mb/s.
 5. The electronic device of claim 1,wherein the first tiled window and the second tiled window have a commonsize.
 6. The electronic device of claim 1, wherein a size of the firsttiled window is at least two times a size of the second tiled window. 7.The electronic device of claim 1, wherein, in response to the interfacecircuit receiving user-interface activity information, associated with aportable electronic device, which specifies a selection of a contentsource, the electronic device is configured to: provide, via theinterface circuit, a request for the content source for a third videvideo stream; and receive a third input data stream that comprisesframes of the third video content, wherein the third input data streamcorrespond to the high-definition format; wherein the output data streamfurther comprises frames of scaled third video content and the displayinstructions further specify a third tiled window associated with thescaled third video content; and wherein the scaled third video contenthas the high-definition format and comprises the component channels. 8.The electronic device of claim 7, wherein a size of the first tiledwindow is at least four times a size of the second tiled window and isat least four times a size of the third tiled window.
 9. Anon-transitory computer-readable storage medium for use in conjunctionwith an electronic device, the computer-readable storage medium storinga program module that, when executed by the electronic device, toprovide an output data stream by causing the electronic device toperform one or more operations comprising: receiving at least two inputdata streams that comprise full frames of first video content and fullframes of second video content, wherein the input data streamscorrespond to a high-definition format; generating, via one or moreintegrated circuits and dedicated memory in the electronic device forvideo processing that has direct memory access, the output data stream,having a data rate that is less than a threshold value associated withswapping data in memory in the electronic device and that is compatiblewith a high-definition multimedia-interface (HDMT) format, and whichcomprises temporally multiplexed frames of scaled first video contentand frames of scaled second video content, and display instructions thatspecify a first tiled window associated with the scaled first videocontent and a second tiled window associated with the scaled secondvideo content, wherein the scaled first video content and the scaledsecond video content have the high-definition format and comprisecomponent channels, and wherein the generating comprises compression ofthe first video content to create the first scaled video content andcompression of the second video content to create the second scaledvideo content; and providing, via an interface circuit in the electronicdevice, the output data stream for a display device, whereinuncompressed video is stored in the dedicated memory and compressedvideo is stored is second memory in the electronic device, and whereinthe second memory stores an operating system of the electronic device,wherein the generating comprises generating the scaled first videocontent by scaling the first video content based on a size of the firsttiled window, and generating the scaled second video content by scalingthe second video content based on a size of the second tiled window. 10.The non-transitory computer-readable storage medium of claim 9, whereinone of the input data streams is encoded; and wherein the generating ofthe output data stream comprises decoding and decompressing the one ofthe input data streams.
 11. The non-transitory computer-readable storagemedium of claim 9, wherein the memory comprises DDR3 and the thresholdvalue is 25 Mb/s.
 12. The non-transitory computer-readable storagemedium of claim 9, wherein the first tiled window and the second tiledwindow have a common size.
 13. The non-transitory computer-readablestorage medium of claim 9, wherein a size of the first tiled window isat least two times a size of the second tiled window.
 14. Thenon-transitory computer-readable storage medium of claim 9, wherein, inresponse to the interface circuit receiving user-interface activityinformation, associated with a portable electronic device, whichspecifies a selection of a content source, the one or more operationscomprise: providing, via the interface circuit, a request for thecontent source for a third video stream; and receiving a third inputdata stream that comprises frames of the third video content, whereinthe third input data stream correspond to the high-definition format;wherein the output data stream further comprises frames of scaled thirdvideo content and the display instructions further specify a third tiledwindow associated with the scaled third video content; and wherein thescaled third video content has the high-definition format and comprisesthe component channels.
 15. The non-transitory computer-readable storagemedium of claim 14, wherein a size of the first tiled window is at leastfour times a size of the second tiled window and is at least four timesa size of the third tiled window.
 16. A method for providing an outputdata stream, wherein the method comprises: by an electronic device:receiving at least two input data streams that comprise full frames offirst video content and full frames of second video content, wherein theinput data streams correspond to a high-definition format; generating,via one or more integrated circuits and dedicated memory in theelectronic device for video processing that has direct memory access,the output data stream, having a data rate that is less than a thresholdvalue associated with swapping data in memory in the electronic deviceand that is compatible with a high-definition multimedia-interface(HDMI) format, and which comprises temporally multiplexed frames ofscaled first video content and frames of scaled second video content,and display instructions that specify a first tiled window associatedwith the scaled first video content and a second tiled window associatedwith the scaled second video content, wherein the scaled first videocontent and the scaled second video content have the high-definitionformat and comprise component channels, and wherein generating comprisescompression of the first video content to create the first scaled videocontent and compression of the second video content to create the secondscaled video content; and providing, via an interface circuit in theelectronic device, the output data stream for a display device, whereinuncompressed video is stored in the dedicated memory and compressedvideo is stored is second memory in the electronic device, and whereinthe second memory stores an operating system of the electronic device,wherein the generating comprises generating the scaled first videocontent by scaling the first video content based on a size of the firsttiled window, and generating the scaled second video content by scalingthe second video content based on a size of the second tiled window. 17.The method of claim 16, wherein the memory comprises DDR3 and thethreshold value is 25 Mb/s.